[ 0.000000] microcode: microcode updated early to revision 0x710, date = 2013-06-17 [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.10.0-693.5.2.el7_lustre.x86_64 (jenkins@trevis-307-el7-x8664-1.trevis.hpdd.intel.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Wed Nov 8 08:06:20 UTC 2017 [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.5.2.el7_lustre.x86_64 root=UUID=2fbe2fc1-ea16-409a-8e17-89fecbe37191 ro crashkernel=320M console=tty0 LANG=en_US.UTF-8 console=ttyS0,115200 net.ifnames=0 [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008d3ff] usable [ 0.000000] BIOS-e820: [mem 0x000000000008d400-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bb3c6fff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bb3c7000-0x00000000bdd2efff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000bdd2f000-0x00000000bddccfff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x00000000bddcd000-0x00000000bdea0fff] ACPI data [ 0.000000] BIOS-e820: [mem 0x00000000bdea1000-0x00000000bdf2efff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x00000000bdf2f000-0x00000000bdfabfff] ACPI data [ 0.000000] BIOS-e820: [mem 0x00000000bdfac000-0x00000000bdffffff] usable [ 0.000000] BIOS-e820: [mem 0x00000000be000000-0x00000000cfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed19000-0x00000000fed19fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ffa20000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000083fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.6 present. [ 0.000000] DMI: Intel Corporation S2600GZ ........../S2600GZ, BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013 [ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved [ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable [ 0.000000] e820: last_pfn = 0x840000 max_arch_pfn = 0x400000000 [ 0.000000] MTRR default type: uncachable [ 0.000000] MTRR fixed ranges enabled: [ 0.000000] 00000-9FFFF write-back [ 0.000000] A0000-BFFFF uncachable [ 0.000000] C0000-DBFFF write-through [ 0.000000] DC000-DFFFF write-protect [ 0.000000] E0000-E7FFF uncachable [ 0.000000] E8000-FFFFF write-protect [ 0.000000] MTRR variable ranges enabled: [ 0.000000] 0 base 000000000000 mask 3FFF80000000 write-back [ 0.000000] 1 base 000080000000 mask 3FFFC0000000 write-back [ 0.000000] 2 base 000100000000 mask 3FFF00000000 write-back [ 0.000000] 3 base 000200000000 mask 3FFE00000000 write-back [ 0.000000] 4 base 000400000000 mask 3FFC00000000 write-back [ 0.000000] 5 base 000800000000 mask 3FFFC0000000 write-back [ 0.000000] 6 base 0000FF800000 mask 3FFFFF800000 write-protect [ 0.000000] 7 disabled [ 0.000000] 8 disabled [ 0.000000] 9 disabled [ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 [ 0.000000] e820: last_pfn = 0xbe000 max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000fcd80-0x000fcd8f] mapped at [ffff8800000fcd80] [ 0.000000] Base memory trampoline at [ffff880000087000] 87000 size 24576 [ 0.000000] Using GB pages for direct mapping [ 0.000000] BRK [0x01fe9000, 0x01fe9fff] PGTABLE [ 0.000000] BRK [0x01fea000, 0x01feafff] PGTABLE [ 0.000000] BRK [0x01feb000, 0x01febfff] PGTABLE [ 0.000000] BRK [0x01fec000, 0x01fecfff] PGTABLE [ 0.000000] RAMDISK: [mem 0x34ad1000-0x36560fff] [ 0.000000] Early table checksum verification disabled [ 0.000000] ACPI: RSDP 00000000000f0410 00024 (v02 INTEL ) [ 0.000000] ACPI: XSDT 00000000bdfa9d98 000BC (v01 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: FACP 00000000bdfa9918 000F4 (v04 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aControlBlock: 32, using default 16 (20130517/tbfadt-672) [ 0.000000] ACPI: DSDT 00000000bdf8f018 18762 (v02 INTEL S2600GZ 00000008 INTL 20100331) [ 0.000000] ACPI: FACS 00000000bdfa9f40 00040 [ 0.000000] ACPI: APIC 00000000bdfa8018 00BAA (v03 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: SPMI 00000000bdfabb18 00040 (v05 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: MCFG 00000000bdfaba98 0003C (v01 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: WDDT 00000000bdfabf18 00040 (v01 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: SRAT 00000000bdf8cc18 002A8 (v03 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: SLIT 00000000bdfabe98 00030 (v01 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: MSCT 00000000bdfaae18 00090 (v01 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: HPET 00000000bdfabe18 00038 (v01 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: SSDT 00000000bdfabd18 0002B (v02 INTEL S2600GZ 00001000 INTL 20100331) [ 0.000000] ACPI: SSDT 00000000bddcd018 D30C8 (v02 INTEL S2600GZ 00004000 INTL 20100331) [ 0.000000] ACPI: SPCR 00000000bdfabd98 00050 (v01 INTEL S2600GZ 06222004 INTL 20090903) [ 0.000000] ACPI: HEST 00000000bdf8ef18 000A8 (v01 INTEL S2600GZ 00000001 INTL 00000001) [ 0.000000] ACPI: BERT 00000000bdfabc18 00030 (v01 INTEL S2600GZ 00000001 INTL 00000001) [ 0.000000] ACPI: ERST 00000000bdf8ec98 00230 (v01 INTEL S2600GZ 00000001 INTL 00000001) [ 0.000000] ACPI: EINJ 00000000bdfa9798 00130 (v01 INTEL S2600GZ 00000001 INTL 00000001) [ 0.000000] ACPI: SSDT 00000000bdf8a018 01729 (v02 INTEL S2600GZ 00000002 INTL 20100331) [ 0.000000] ACPI: SSDT 00000000bdfabc98 00045 (v02 INTEL S2600GZ 00000001 INTL 20100331) [ 0.000000] ACPI: SSDT 00000000bdf8de18 00181 (v02 INTEL S2600GZ 00000003 INTL 20100331) [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] SRAT: PXM 0 -> APIC 0x00 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x01 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x02 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x03 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x04 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x05 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x06 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x07 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x08 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x09 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x0a -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x0b -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x0c -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x0d -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x0e -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 0x0f -> Node 0 [ 0.000000] SRAT: PXM 1 -> APIC 0x20 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x21 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x22 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x23 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x24 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x25 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x26 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x27 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x28 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x29 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x2a -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x2b -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x2c -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x2d -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x2e -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 0x2f -> Node 1 [ 0.000000] SRAT: Node 0 PXM 0 [mem 0x00000000-0xbfffffff] [ 0.000000] SRAT: Node 0 PXM 0 [mem 0x100000000-0x43fffffff] [ 0.000000] SRAT: Node 1 PXM 1 [mem 0x440000000-0x83fffffff] [ 0.000000] NUMA: Initialized distance table, cnt=2 [ 0.000000] NUMA: Node 0 [mem 0x00000000-0xbfffffff] + [mem 0x100000000-0x43fffffff] -> [mem 0x00000000-0x43fffffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x43ffd9000-0x43fffffff] [ 0.000000] NODE_DATA(1) allocated [mem 0x83ffd6000-0x83fffcfff] [ 0.000000] Reserving 320MB of memory at 512MB for crashkernel (System RAM: 32691MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x00001000-0x00ffffff] [ 0.000000] DMA32 [mem 0x01000000-0xffffffff] [ 0.000000] Normal [mem 0x100000000-0x83fffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0008cfff] [ 0.000000] node 0: [mem 0x00100000-0xbb3c6fff] [ 0.000000] node 0: [mem 0xbdfac000-0xbdffffff] [ 0.000000] node 0: [mem 0x100000000-0x43fffffff] [ 0.000000] node 1: [mem 0x440000000-0x83fffffff] [ 0.000000] Initmem setup node 0 [mem 0x00001000-0x43fffffff] [ 0.000000] On node 0 totalpages: 4174759 [ 0.000000] DMA zone: 64 pages used for memmap [ 0.000000] DMA zone: 21 pages reserved [ 0.000000] DMA zone: 3980 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 11921 pages used for memmap [ 0.000000] DMA32 zone: 762907 pages, LIFO batch:31 [ 0.000000] Normal zone: 53248 pages used for memmap [ 0.000000] Normal zone: 3407872 pages, LIFO batch:31 [ 0.000000] Initmem setup node 1 [mem 0x440000000-0x83fffffff] [ 0.000000] On node 1 totalpages: 4194304 [ 0.000000] Normal zone: 65536 pages used for memmap [ 0.000000] Normal zone: 4194304 pages, LIFO batch:31 [ 0.000000] ACPI: PM-Timer IO Port: 0x408 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x0c] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0e] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x20] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x22] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x24] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x26] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x28] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x2a] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x2c] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x2e] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x01] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x03] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x05] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x07] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x09] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0x0b] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0x0d] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0x0f] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0x21] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0x23] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x25] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x27] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x29] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x2b] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x2d] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x2f] enabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x20] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x21] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x22] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x23] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x24] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x25] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x26] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x27] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x28] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x29] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x2a] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x2b] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x2c] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x2d] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x2e] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x2f] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x30] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x31] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x32] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x33] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x34] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x35] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x36] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x37] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x38] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x39] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x3a] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x3b] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x3c] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x3d] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x3e] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x3f] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x40] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x41] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x42] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x43] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x44] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x45] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x46] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x47] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x48] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x49] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x4a] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x4b] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x4c] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x4d] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x4e] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x4f] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x50] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x51] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x52] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x53] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x54] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x55] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x56] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x57] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x58] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x59] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x5a] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x5b] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x5c] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x5d] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x5e] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x5f] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x60] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x61] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x62] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x63] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x64] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x65] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x66] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x67] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x68] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x69] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x6a] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x6b] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x6c] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x6d] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x6e] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x6f] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x70] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x71] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x72] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x73] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x74] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x75] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x76] lapic_id[0xff] disabled) [ 0.000000] ACPI: LAPIC (acpi_id[0x77] lapic_id[0xff] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x00] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x01] uid[0x01] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x02] uid[0x02] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x03] uid[0x03] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x04] uid[0x04] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x05] uid[0x05] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x06] uid[0x06] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x07] uid[0x07] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x08] uid[0x08] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x09] uid[0x09] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x0a] uid[0x0a] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x0b] uid[0x0b] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x0c] uid[0x0c] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x0d] uid[0x0d] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x0e] uid[0x0e] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x0f] uid[0x0f] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x10] uid[0x10] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x11] uid[0x11] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x12] uid[0x12] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x13] uid[0x13] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x14] uid[0x14] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x15] uid[0x15] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x16] uid[0x16] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x17] uid[0x17] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x18] uid[0x18] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x19] uid[0x19] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x1a] uid[0x1a] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x1b] uid[0x1b] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x1c] uid[0x1c] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x1d] uid[0x1d] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x1e] uid[0x1e] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x1f] uid[0x1f] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x20] uid[0x20] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x21] uid[0x21] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x22] uid[0x22] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x23] uid[0x23] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x24] uid[0x24] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x25] uid[0x25] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x26] uid[0x26] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x27] uid[0x27] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x28] uid[0x28] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x29] uid[0x29] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x2a] uid[0x2a] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x2b] uid[0x2b] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x2c] uid[0x2c] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x2d] uid[0x2d] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x2e] uid[0x2e] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x2f] uid[0x2f] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x30] uid[0x30] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x31] uid[0x31] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x32] uid[0x32] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x33] uid[0x33] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x34] uid[0x34] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x35] uid[0x35] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x36] uid[0x36] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x37] uid[0x37] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x38] uid[0x38] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x39] uid[0x39] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x3a] uid[0x3a] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x3b] uid[0x3b] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x3c] uid[0x3c] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x3d] uid[0x3d] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x3e] uid[0x3e] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x3f] uid[0x3f] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x40] uid[0x40] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x41] uid[0x41] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x42] uid[0x42] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x43] uid[0x43] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x44] uid[0x44] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x45] uid[0x45] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x46] uid[0x46] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x47] uid[0x47] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x48] uid[0x48] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x49] uid[0x49] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x4a] uid[0x4a] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x4b] uid[0x4b] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x4c] uid[0x4c] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x4d] uid[0x4d] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x4e] uid[0x4e] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x4f] uid[0x4f] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x50] uid[0x50] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x51] uid[0x51] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x52] uid[0x52] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x53] uid[0x53] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x54] uid[0x54] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x55] uid[0x55] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x56] uid[0x56] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x57] uid[0x57] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x58] uid[0x58] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x59] uid[0x59] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x5a] uid[0x5a] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x5b] uid[0x5b] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x5c] uid[0x5c] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x5d] uid[0x5d] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x5e] uid[0x5e] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x5f] uid[0x5f] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x60] uid[0x60] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x61] uid[0x61] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x62] uid[0x62] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x63] uid[0x63] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x64] uid[0x64] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x65] uid[0x65] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x66] uid[0x66] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x67] uid[0x67] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x68] uid[0x68] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x69] uid[0x69] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x6a] uid[0x6a] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x6b] uid[0x6b] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x6c] uid[0x6c] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x6d] uid[0x6d] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x6e] uid[0x6e] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x6f] uid[0x6f] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x70] uid[0x70] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x71] uid[0x71] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x72] uid[0x72] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x73] uid[0x73] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x74] uid[0x74] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x75] uid[0x75] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x76] uid[0x76] disabled) [ 0.000000] ACPI: X2APIC (apic_id[0x77] uid[0x77] disabled) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: IOAPIC (id[0x01] address[0xfec3f000] gsi_base[24]) [ 0.000000] IOAPIC[1]: apic_id 1, version 32, address 0xfec3f000, GSI 24-47 [ 0.000000] ACPI: IOAPIC (id[0x02] address[0xfec7f000] gsi_base[48]) [ 0.000000] IOAPIC[2]: apic_id 2, version 32, address 0xfec7f000, GSI 48-71 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: IRQ0 used by override. [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000 [ 0.000000] smpboot: Allowing 240 CPUs, 208 hotplug CPUs [ 0.000000] PM: Registered nosave memory: [mem 0x0008d000-0x0008dfff] [ 0.000000] PM: Registered nosave memory: [mem 0x0008e000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000dffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbb3c7000-0xbdd2efff] [ 0.000000] PM: Registered nosave memory: [mem 0xbdd2f000-0xbddccfff] [ 0.000000] PM: Registered nosave memory: [mem 0xbddcd000-0xbdea0fff] [ 0.000000] PM: Registered nosave memory: [mem 0xbdea1000-0xbdf2efff] [ 0.000000] PM: Registered nosave memory: [mem 0xbdf2f000-0xbdfabfff] [ 0.000000] PM: Registered nosave memory: [mem 0xbe000000-0xcfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xd0000000-0xfebfffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfec00000-0xfec00fff] [ 0.000000] PM: Registered nosave memory: [mem 0xfec01000-0xfed18fff] [ 0.000000] PM: Registered nosave memory: [mem 0xfed19000-0xfed19fff] [ 0.000000] PM: Registered nosave memory: [mem 0xfed1a000-0xfed1bfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed1ffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfed20000-0xfedfffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfee00000-0xfee00fff] [ 0.000000] PM: Registered nosave memory: [mem 0xfee01000-0xffa1ffff] [ 0.000000] PM: Registered nosave memory: [mem 0xffa20000-0xffffffff] [ 0.000000] e820: [mem 0xd0000000-0xfebfffff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on bare hardware [ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:240 nr_cpu_ids:240 nr_node_ids:2 [ 0.000000] PERCPU: Embedded 33 pages/cpu @ffff88042e000000 s97048 r8192 d29928 u262144 [ 0.000000] pcpu-alloc: s97048 r8192 d29928 u262144 alloc=1*2097152 [ 0.000000] pcpu-alloc: [0] 000 001 002 003 004 005 006 007 [ 0.000000] pcpu-alloc: [0] 016 017 018 019 020 021 022 023 [ 0.000000] pcpu-alloc: [0] 032 034 036 038 040 042 044 046 [ 0.000000] pcpu-alloc: [0] 048 050 052 054 056 058 060 062 [ 0.000000] pcpu-alloc: [0] 064 066 068 070 072 074 076 078 [ 0.000000] pcpu-alloc: [0] 080 082 084 086 088 090 092 094 [ 0.000000] pcpu-alloc: [0] 096 098 100 102 104 106 108 110 [ 0.000000] pcpu-alloc: [0] 112 114 116 118 120 122 124 126 [ 0.000000] pcpu-alloc: [0] 128 130 132 134 136 138 140 142 [ 0.000000] pcpu-alloc: [0] 144 146 148 150 152 154 156 158 [ 0.000000] pcpu-alloc: [0] 160 162 164 166 168 170 172 174 [ 0.000000] pcpu-alloc: [0] 176 178 180 182 184 186 188 190 [ 0.000000] pcpu-alloc: [0] 192 194 196 198 200 202 204 206 [ 0.000000] pcpu-alloc: [0] 208 210 212 214 216 218 220 222 [ 0.000000] pcpu-alloc: [0] 224 226 228 230 232 234 236 238 [ 0.000000] pcpu-alloc: [1] 008 009 010 011 012 013 014 015 [ 0.000000] pcpu-alloc: [1] 024 025 026 027 028 029 030 031 [ 0.000000] pcpu-alloc: [1] 033 035 037 039 041 043 045 047 [ 0.000000] pcpu-alloc: [1] 049 051 053 055 057 059 061 063 [ 0.000000] pcpu-alloc: [1] 065 067 069 071 073 075 077 079 [ 0.000000] pcpu-alloc: [1] 081 083 085 087 089 091 093 095 [ 0.000000] pcpu-alloc: [1] 097 099 101 103 105 107 109 111 [ 0.000000] pcpu-alloc: [1] 113 115 117 119 121 123 125 127 [ 0.000000] pcpu-alloc: [1] 129 131 133 135 137 139 141 143 [ 0.000000] pcpu-alloc: [1] 145 147 149 151 153 155 157 159 [ 0.000000] pcpu-alloc: [1] 161 163 165 167 169 171 173 175 [ 0.000000] pcpu-alloc: [1] 177 179 181 183 185 187 189 191 [ 0.000000] pcpu-alloc: [1] 193 195 197 199 201 203 205 207 [ 0.000000] pcpu-alloc: [1] 209 211 213 215 217 219 221 223 [ 0.000000] pcpu-alloc: [1] 225 227 229 231 233 235 237 239 [ 0.000000] Built 2 zonelists in Zone order, mobility grouping on. Total pages: 8238273 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-3.10.0-693.5.2.el7_lustre.x86_64 root=UUID=2fbe2fc1-ea16-409a-8e17-89fecbe37191 ro crashkernel=320M console=tty0 LANG=en_US.UTF-8 console=ttyS0,115200 net.ifnames=0 [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 [ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form [ 0.000000] Memory: 6824980k/34603008k available (6886k kernel code, 1126756k absent, 994016k reserved, 4545k data, 1764k init) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=240, Nodes=2 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=240. [ 0.000000] NR_IRQS:327936 nr_irqs:3160 0 [ 0.000000] Console: colour VGA+ 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [ttyS0] enabled [ 0.000000] allocated 134217728 bytes of page_cgroup [ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups [ 0.000000] Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl [ 0.000000] hpet clockevent registered [ 0.000000] tsc: Fast TSC calibration using PIT [ 0.000000] tsc: Detected 2693.506 MHz processor [ 0.000102] Calibrating delay loop (skipped), value calculated using timer frequency.. 5387.01 BogoMIPS (lpj=2693506) [ 0.012525] pid_max: default: 245760 minimum: 1920 [ 0.018815] Security Framework initialized [ 0.023678] SELinux: Initializing. [ 0.027927] SELinux: Starting in permissive mode [ 0.027928] Yama: becoming mindful. [ 0.034565] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes) [ 0.049824] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes) [ 0.061224] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes) [ 0.069230] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes) [ 0.079665] Initializing cgroup subsys memory [ 0.084858] Initializing cgroup subsys devices [ 0.090101] Initializing cgroup subsys freezer [ 0.095351] Initializing cgroup subsys net_cls [ 0.100595] Initializing cgroup subsys blkio [ 0.105646] Initializing cgroup subsys perf_event [ 0.111221] Initializing cgroup subsys hugetlb [ 0.116465] Initializing cgroup subsys pids [ 0.121417] Initializing cgroup subsys net_prio [ 0.126912] CPU: Physical Processor ID: 0 [ 0.131669] CPU: Processor Core ID: 0 [ 0.136043] mce: CPU supports 20 MCE banks [ 0.140899] CPU0: Thermal monitoring enabled (TM1) [ 0.146571] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0 [ 0.152983] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32 [ 0.159587] tlb_flushall_shift: 6 [ 0.163728] Freeing SMP alternatives: 24k freed [ 0.171559] ACPI: Core revision 20130517 [ 0.213463] ACPI: All ACPI Tables successfully acquired [ 0.235137] ftrace: allocating 26589 entries in 104 pages [ 0.254193] smpboot: Max logical packages: 30 [ 0.259916] IRQ remapping doesn't support X2APIC mode, disable x2apic. [ 0.267567] Switched APIC routing to physical flat. [ 0.273918] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.290909] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz (fam: 06, model: 2d, stepping: 07) [ 0.302035] TSC deadline timer enabled [ 0.302068] Performance Events: PEBS fmt1+, 16-deep LBR, SandyBridge events, full-width counters, Intel PMU driver. [ 0.314634] ... version: 3 [ 0.319385] ... bit width: 48 [ 0.324236] ... generic registers: 4 [ 0.328984] ... value mask: 0000ffffffffffff [ 0.335195] ... max period: 00007fffffffffff [ 0.341404] ... fixed-purpose events: 3 [ 0.346153] ... event mask: 000000070000000f [ 0.377308] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter. [ 0.361628] smpboot: Booting Node 0, Processors #1 #2 #3 #4 #5 #6 #7 OK [ 0.414454] smpboot: Booting Node 1, Processors #8 #9 #10 #11 #12 #13 #14 #15 OK [ 0.541434] smpboot: Booting Node 0, Processors #16 #17 #18 #19 #20 #21 #22 #23 OK [ 0.576102] smpboot: Booting Node 1, Processors #24 #25 #26 #27 #28 #29 #30 #31 [ 0.616574] Brought up 32 CPUs [ 0.620725] smpboot: Total of 32 processors activated (172468.03 BogoMIPS) [ 0.744372] node 0 initialised, 2813971 pages in 62ms [ 0.761561] node 1 initialised, 3600339 pages in 74ms [ 0.768773] devtmpfs: initialized [ 0.781884] EVM: security.selinux [ 0.785877] EVM: security.ima [ 0.789479] EVM: security.capability [ 0.794053] PM: Registering ACPI NVS region [mem 0xbdd2f000-0xbddccfff] (647168 bytes) [ 0.803432] PM: Registering ACPI NVS region [mem 0xbdea1000-0xbdf2efff] (581632 bytes) [ 0.815270] atomic64 test passed for x86-64 platform with CX8 and with SSE [ 0.823253] pinctrl core: initialized pinctrl subsystem [ 0.829653] RTC time: 2:01:19, date: 11/16/17 [ 0.835420] NET: Registered protocol family 16 [ 0.841126] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it [ 0.850090] ACPI: bus type PCI registered [ 0.854849] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.862951] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xc0000000-0xcfffffff] (base 0xc0000000) [ 0.873858] PCI: MMCONFIG at [mem 0xc0000000-0xcfffffff] reserved in E820 [ 0.881781] PCI: Using configuration type 1 for base access [ 0.888370] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.906646] ACPI: Added _OSI(Module Device) [ 0.911610] ACPI: Added _OSI(Processor Device) [ 0.916864] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.922408] ACPI: Added _OSI(Processor Aggregator Device) [ 0.946492] ACPI: EC: Look up EC in DSDT [ 0.964506] ACPI: Executed 1 blocks of module-level executable AML code [ 1.236255] [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored [ 1.267015] ACPI: Interpreter enabled [ 1.271425] ACPI: (supports S0 S1 S5) [ 1.275803] ACPI: Using IOAPIC for interrupt routing [ 1.281743] HEST: Table parsing has been initialized. [ 1.287685] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 1.351267] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e]) [ 1.358477] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 1.368612] acpi PNP0A08:00: _OSC: platform does not support [AER] [ 1.376210] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.385269] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.395184] acpi PNP0A08:00: ignoring host bridge window [mem 0x000d0000-0x000d3fff window] (conflicts with Adapter ROM [mem 0x000c8000-0x000d1bff]) [ 1.410564] acpi PNP0A08:00: ignoring host bridge window [mem 0x000d8000-0x000dbfff window] (conflicts with Adapter ROM [mem 0x000d2000-0x000d8bff]) [ 1.426300] PCI host bridge to bus 0000:00 [ 1.431170] pci_bus 0000:00: root bus resource [bus 00-7e] [ 1.437592] pci_bus 0000:00: root bus resource [io 0x0000-0xbfff window] [ 1.445473] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 1.454334] pci_bus 0000:00: root bus resource [mem 0x000c0000-0x000c3fff window] [ 1.463198] pci_bus 0000:00: root bus resource [mem 0x000c4000-0x000c7fff window] [ 1.472055] pci_bus 0000:00: root bus resource [mem 0x000c8000-0x000cbfff window] [ 1.480915] pci_bus 0000:00: root bus resource [mem 0x000cc000-0x000cffff window] [ 1.489781] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff window] [ 1.498643] pci_bus 0000:00: root bus resource [mem 0x000dc000-0x000dffff window] [ 1.507503] pci_bus 0000:00: root bus resource [mem 0x000e0000-0x000e3fff window] [ 1.516366] pci_bus 0000:00: root bus resource [mem 0x000e4000-0x000e7fff window] [ 1.525225] pci_bus 0000:00: root bus resource [mem 0x000e8000-0x000ebfff window] [ 1.534089] pci_bus 0000:00: root bus resource [mem 0x000ec000-0x000effff window] [ 1.542952] pci_bus 0000:00: root bus resource [mem 0x000f0000-0x000fffff window] [ 1.551810] pci_bus 0000:00: root bus resource [mem 0xd0000000-0xebffffff window] [ 1.560670] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x38007fffffff window] [ 1.570324] pci 0000:00:00.0: [8086:3c00] type 00 class 0x060000 [ 1.570421] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold [ 1.570572] pci 0000:00:01.0: [8086:3c02] type 01 class 0x060400 [ 1.570673] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold [ 1.570775] pci 0000:00:01.0: System wakeup disabled by ACPI [ 1.577447] pci 0000:00:01.1: [8086:3c03] type 01 class 0x060400 [ 1.577547] pci 0000:00:01.1: PME# supported from D0 D3hot D3cold [ 1.577648] pci 0000:00:01.1: System wakeup disabled by ACPI [ 1.584330] pci 0000:00:02.0: [8086:3c04] type 01 class 0x060400 [ 1.584431] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold [ 1.584528] pci 0000:00:02.0: System wakeup disabled by ACPI [ 1.591199] pci 0000:00:02.2: [8086:3c06] type 01 class 0x060400 [ 1.591299] pci 0000:00:02.2: PME# supported from D0 D3hot D3cold [ 1.591396] pci 0000:00:02.2: System wakeup disabled by ACPI [ 1.598076] pci 0000:00:03.0: [8086:3c08] type 01 class 0x060400 [ 1.598176] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold [ 1.598273] pci 0000:00:03.0: System wakeup disabled by ACPI [ 1.604943] pci 0000:00:04.0: [8086:3c20] type 00 class 0x088000 [ 1.604967] pci 0000:00:04.0: reg 0x10: [mem 0xebf90000-0xebf93fff 64bit] [ 1.605167] pci 0000:00:04.1: [8086:3c21] type 00 class 0x088000 [ 1.605190] pci 0000:00:04.1: reg 0x10: [mem 0xebf80000-0xebf83fff 64bit] [ 1.605384] pci 0000:00:04.2: [8086:3c22] type 00 class 0x088000 [ 1.605406] pci 0000:00:04.2: reg 0x10: [mem 0xebf70000-0xebf73fff 64bit] [ 1.605600] pci 0000:00:04.3: [8086:3c23] type 00 class 0x088000 [ 1.605622] pci 0000:00:04.3: reg 0x10: [mem 0xebf60000-0xebf63fff 64bit] [ 1.605829] pci 0000:00:04.4: [8086:3c24] type 00 class 0x088000 [ 1.605850] pci 0000:00:04.4: reg 0x10: [mem 0xebf50000-0xebf53fff 64bit] [ 1.606053] pci 0000:00:04.5: [8086:3c25] type 00 class 0x088000 [ 1.606075] pci 0000:00:04.5: reg 0x10: [mem 0xebf40000-0xebf43fff 64bit] [ 1.606267] pci 0000:00:04.6: [8086:3c26] type 00 class 0x088000 [ 1.606289] pci 0000:00:04.6: reg 0x10: [mem 0xebf30000-0xebf33fff 64bit] [ 1.606482] pci 0000:00:04.7: [8086:3c27] type 00 class 0x088000 [ 1.606504] pci 0000:00:04.7: reg 0x10: [mem 0xebf20000-0xebf23fff 64bit] [ 1.606701] pci 0000:00:05.0: [8086:3c28] type 00 class 0x088000 [ 1.606887] pci 0000:00:05.2: [8086:3c2a] type 00 class 0x088000 [ 1.607075] pci 0000:00:05.4: [8086:3c2c] type 00 class 0x080020 [ 1.607092] pci 0000:00:05.4: reg 0x10: [mem 0xd0f60000-0xd0f60fff] [ 1.607297] pci 0000:00:11.0: [8086:1d3e] type 01 class 0x060400 [ 1.607417] pci 0000:00:11.0: PME# supported from D0 D3hot D3cold [ 1.607570] pci 0000:00:16.0: [8086:1d3a] type 00 class 0x078000 [ 1.607595] pci 0000:00:16.0: reg 0x10: [mem 0xd0f50000-0xd0f5000f 64bit] [ 1.607684] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold [ 1.607818] pci 0000:00:16.1: [8086:1d3b] type 00 class 0x078000 [ 1.607844] pci 0000:00:16.1: reg 0x10: [mem 0xd0f40000-0xd0f4000f 64bit] [ 1.607935] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold [ 1.608076] pci 0000:00:1a.0: [8086:1d2d] type 00 class 0x0c0320 [ 1.608100] pci 0000:00:1a.0: reg 0x10: [mem 0xd0f20000-0xd0f203ff] [ 1.608211] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold [ 1.608318] pci 0000:00:1a.0: System wakeup disabled by ACPI [ 1.615001] pci 0000:00:1c.0: [8086:1d10] type 01 class 0x060400 [ 1.615136] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold [ 1.615228] pci 0000:00:1c.0: System wakeup disabled by ACPI [ 1.621905] pci 0000:00:1c.7: [8086:1d1e] type 01 class 0x060400 [ 1.622009] pci 0000:00:1c.7: PME# supported from D0 D3hot D3cold [ 1.622100] pci 0000:00:1c.7: System wakeup disabled by ACPI [ 1.628777] pci 0000:00:1d.0: [8086:1d26] type 00 class 0x0c0320 [ 1.628801] pci 0000:00:1d.0: reg 0x10: [mem 0xd0f10000-0xd0f103ff] [ 1.628913] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold [ 1.629018] pci 0000:00:1d.0: System wakeup disabled by ACPI [ 1.635686] pci 0000:00:1e.0: [8086:244e] type 01 class 0x060401 [ 1.635832] pci 0000:00:1e.0: System wakeup disabled by ACPI [ 1.642499] pci 0000:00:1f.0: [8086:1d41] type 00 class 0x060100 [ 1.642748] pci 0000:00:1f.2: [8086:1d02] type 00 class 0x010601 [ 1.642771] pci 0000:00:1f.2: reg 0x10: [io 0x4070-0x4077] [ 1.642782] pci 0000:00:1f.2: reg 0x14: [io 0x4060-0x4063] [ 1.642795] pci 0000:00:1f.2: reg 0x18: [io 0x4050-0x4057] [ 1.642808] pci 0000:00:1f.2: reg 0x1c: [io 0x4040-0x4043] [ 1.642820] pci 0000:00:1f.2: reg 0x20: [io 0x4020-0x403f] [ 1.642832] pci 0000:00:1f.2: reg 0x24: [mem 0xd0f00000-0xd0f007ff] [ 1.642891] pci 0000:00:1f.2: PME# supported from D3hot [ 1.643016] pci 0000:00:1f.3: [8086:1d22] type 00 class 0x0c0500 [ 1.643038] pci 0000:00:1f.3: reg 0x10: [mem 0xebf10000-0xebf100ff 64bit] [ 1.643066] pci 0000:00:1f.3: reg 0x20: [io 0x4000-0x401f] [ 1.643281] pci 0000:00:01.0: PCI bridge to [bus 01] [ 1.649275] acpiphp: Slot [2] registered [ 1.654004] pci 0000:02:00.0: [8086:1521] type 00 class 0x020000 [ 1.654023] pci 0000:02:00.0: reg 0x10: [mem 0xd0b60000-0xd0b7ffff] [ 1.654042] pci 0000:02:00.0: reg 0x18: [io 0x3060-0x307f] [ 1.654053] pci 0000:02:00.0: reg 0x1c: [mem 0xd0bb0000-0xd0bb3fff] [ 1.654177] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold [ 1.654211] pci 0000:02:00.0: reg 0x184: [mem 0xd0ca0000-0xd0ca3fff] [ 1.654215] pci 0000:02:00.0: VF(n) BAR0 space: [mem 0xd0ca0000-0xd0cbffff] (contains BAR0 for 8 VFs) [ 1.665041] pci 0000:02:00.0: reg 0x190: [mem 0xd0c80000-0xd0c83fff] [ 1.665045] pci 0000:02:00.0: VF(n) BAR3 space: [mem 0xd0c80000-0xd0c9ffff] (contains BAR3 for 8 VFs) [ 1.676000] pci 0000:02:00.1: [8086:1521] type 00 class 0x020000 [ 1.676018] pci 0000:02:00.1: reg 0x10: [mem 0xd0b40000-0xd0b5ffff] [ 1.676036] pci 0000:02:00.1: reg 0x18: [io 0x3040-0x305f] [ 1.676048] pci 0000:02:00.1: reg 0x1c: [mem 0xd0ba0000-0xd0ba3fff] [ 1.676159] pci 0000:02:00.1: PME# supported from D0 D3hot D3cold [ 1.676186] pci 0000:02:00.1: reg 0x184: [mem 0xd0c60000-0xd0c63fff] [ 1.676190] pci 0000:02:00.1: VF(n) BAR0 space: [mem 0xd0c60000-0xd0c7ffff] (contains BAR0 for 8 VFs) [ 1.687018] pci 0000:02:00.1: reg 0x190: [mem 0xd0c40000-0xd0c43fff] [ 1.687022] pci 0000:02:00.1: VF(n) BAR3 space: [mem 0xd0c40000-0xd0c5ffff] (contains BAR3 for 8 VFs) [ 1.697965] pci 0000:02:00.2: [8086:1521] type 00 class 0x020000 [ 1.697984] pci 0000:02:00.2: reg 0x10: [mem 0xd0b20000-0xd0b3ffff] [ 1.698003] pci 0000:02:00.2: reg 0x18: [io 0x3020-0x303f] [ 1.698014] pci 0000:02:00.2: reg 0x1c: [mem 0xd0b90000-0xd0b93fff] [ 1.698125] pci 0000:02:00.2: PME# supported from D0 D3hot D3cold [ 1.698151] pci 0000:02:00.2: reg 0x184: [mem 0xd0c20000-0xd0c23fff] [ 1.698155] pci 0000:02:00.2: VF(n) BAR0 space: [mem 0xd0c20000-0xd0c3ffff] (contains BAR0 for 8 VFs) [ 1.708984] pci 0000:02:00.2: reg 0x190: [mem 0xd0c00000-0xd0c03fff] [ 1.708988] pci 0000:02:00.2: VF(n) BAR3 space: [mem 0xd0c00000-0xd0c1ffff] (contains BAR3 for 8 VFs) [ 1.719930] pci 0000:02:00.3: [8086:1521] type 00 class 0x020000 [ 1.719948] pci 0000:02:00.3: reg 0x10: [mem 0xd0b00000-0xd0b1ffff] [ 1.719966] pci 0000:02:00.3: reg 0x18: [io 0x3000-0x301f] [ 1.719978] pci 0000:02:00.3: reg 0x1c: [mem 0xd0b80000-0xd0b83fff] [ 1.720089] pci 0000:02:00.3: PME# supported from D0 D3hot D3cold [ 1.720116] pci 0000:02:00.3: reg 0x184: [mem 0xd0be0000-0xd0be3fff] [ 1.720120] pci 0000:02:00.3: VF(n) BAR0 space: [mem 0xd0be0000-0xd0bfffff] (contains BAR0 for 8 VFs) [ 1.730945] pci 0000:02:00.3: reg 0x190: [mem 0xd0bc0000-0xd0bc3fff] [ 1.730949] pci 0000:02:00.3: VF(n) BAR3 space: [mem 0xd0bc0000-0xd0bdffff] (contains BAR3 for 8 VFs) [ 1.743781] pci 0000:00:01.1: PCI bridge to [bus 02-03] [ 1.749915] pci 0000:00:01.1: bridge window [io 0x3000-0x3fff] [ 1.749921] pci 0000:00:01.1: bridge window [mem 0xd0b00000-0xd0cfffff] [ 1.750075] acpiphp: Slot [2-2] registered [ 1.755174] pci 0000:04:00.0: [15b3:1003] type 00 class 0x028000 [ 1.755450] pci 0000:04:00.0: reg 0x10: [mem 0xd0e00000-0xd0efffff 64bit] [ 1.755652] pci 0000:04:00.0: reg 0x18: [mem 0xeb000000-0xeb7fffff 64bit pref] [ 1.757384] pci 0000:00:02.0: PCI bridge to [bus 04] [ 1.763229] pci 0000:00:02.0: bridge window [mem 0xd0e00000-0xd0efffff] [ 1.763237] pci 0000:00:02.0: bridge window [mem 0xeb000000-0xeb7fffff 64bit pref] [ 1.763383] acpiphp: Slot [2-3] registered [ 1.768288] pci 0000:00:02.2: PCI bridge to [bus 05] [ 1.774283] acpiphp: Slot [2-4] registered [ 1.779197] pci 0000:06:00.0: [1000:0087] type 00 class 0x010700 [ 1.779213] pci 0000:06:00.0: reg 0x10: [io 0x2000-0x20ff] [ 1.779227] pci 0000:06:00.0: reg 0x14: [mem 0xd0a40000-0xd0a4ffff 64bit] [ 1.779243] pci 0000:06:00.0: reg 0x1c: [mem 0xd0a00000-0xd0a3ffff 64bit] [ 1.779261] pci 0000:06:00.0: reg 0x30: [mem 0xd0900000-0xd09fffff pref] [ 1.779353] pci 0000:06:00.0: supports D1 D2 [ 1.781167] pci 0000:00:03.0: PCI bridge to [bus 06] [ 1.787012] pci 0000:00:03.0: bridge window [io 0x2000-0x2fff] [ 1.787018] pci 0000:00:03.0: bridge window [mem 0xd0900000-0xd0afffff] [ 1.787125] pci 0000:07:00.0: [8086:1d6b] type 00 class 0x010700 [ 1.787154] pci 0000:07:00.0: reg 0x10: [mem 0xebc00000-0xebc03fff 64bit pref] [ 1.787174] pci 0000:07:00.0: reg 0x18: [mem 0xeb800000-0xebbfffff 64bit pref] [ 1.787188] pci 0000:07:00.0: reg 0x20: [io 0x1000-0x10ff] [ 1.787326] pci 0000:07:00.0: reg 0x164: [mem 0xebc10000-0xebc13fff 64bit pref] [ 1.787331] pci 0000:07:00.0: VF(n) BAR0 space: [mem 0xebc10000-0xebc8bfff 64bit pref] (contains BAR0 for 31 VFs) [ 1.799523] pci 0000:07:00.3: [8086:1d70] type 00 class 0x0c0500 [ 1.799548] pci 0000:07:00.3: reg 0x10: [mem 0xd0d00000-0xd0d00fff] [ 1.799599] pci 0000:07:00.3: reg 0x20: [io 0x1100-0x111f] [ 1.799698] pci 0000:07:00.3: PME# supported from D0 D3hot D3cold [ 1.799802] pci 0000:00:11.0: PCI bridge to [bus 07] [ 1.805636] pci 0000:00:11.0: bridge window [io 0x1000-0x1fff] [ 1.805643] pci 0000:00:11.0: bridge window [mem 0xd0d00000-0xd0dfffff] [ 1.805652] pci 0000:00:11.0: bridge window [mem 0xeb800000-0xebcfffff 64bit pref] [ 1.805765] pci 0000:00:1c.0: PCI bridge to [bus 08] [ 1.811727] pci 0000:09:00.0: [102b:0522] type 00 class 0x030000 [ 1.811762] pci 0000:09:00.0: reg 0x10: [mem 0xea000000-0xeaffffff pref] [ 1.811782] pci 0000:09:00.0: reg 0x14: [mem 0xd0810000-0xd0813fff] [ 1.811803] pci 0000:09:00.0: reg 0x18: [mem 0xd0000000-0xd07fffff] [ 1.811876] pci 0000:09:00.0: reg 0x30: [mem 0xd0800000-0xd080ffff pref] [ 1.812013] pci 0000:09:00.0: System wakeup disabled by ACPI [ 1.820652] pci 0000:00:1c.7: PCI bridge to [bus 09] [ 1.826497] pci 0000:00:1c.7: bridge window [mem 0xd0000000-0xd08fffff] [ 1.826506] pci 0000:00:1c.7: bridge window [mem 0xea000000-0xeaffffff 64bit pref] [ 1.826618] pci 0000:00:1e.0: PCI bridge to [bus 0a] (subtractive decode) [ 1.834509] pci 0000:00:1e.0: bridge window [io 0x0000-0xbfff window] (subtractive decode) [ 1.834513] pci 0000:00:1e.0: bridge window [mem 0x000a0000-0x000bffff window] (subtractive decode) [ 1.834517] pci 0000:00:1e.0: bridge window [mem 0x000c0000-0x000c3fff window] (subtractive decode) [ 1.834520] pci 0000:00:1e.0: bridge window [mem 0x000c4000-0x000c7fff window] (subtractive decode) [ 1.834523] pci 0000:00:1e.0: bridge window [mem 0x000c8000-0x000cbfff window] (subtractive decode) [ 1.834526] pci 0000:00:1e.0: bridge window [mem 0x000cc000-0x000cffff window] (subtractive decode) [ 1.834530] pci 0000:00:1e.0: bridge window [mem 0x000d4000-0x000d7fff window] (subtractive decode) [ 1.834533] pci 0000:00:1e.0: bridge window [mem 0x000dc000-0x000dffff window] (subtractive decode) [ 1.834536] pci 0000:00:1e.0: bridge window [mem 0x000e0000-0x000e3fff window] (subtractive decode) [ 1.834539] pci 0000:00:1e.0: bridge window [mem 0x000e4000-0x000e7fff window] (subtractive decode) [ 1.834543] pci 0000:00:1e.0: bridge window [mem 0x000e8000-0x000ebfff window] (subtractive decode) [ 1.834559] pci 0000:00:1e.0: bridge window [mem 0x000ec000-0x000effff window] (subtractive decode) [ 1.834562] pci 0000:00:1e.0: bridge window [mem 0x000f0000-0x000fffff window] (subtractive decode) [ 1.834565] pci 0000:00:1e.0: bridge window [mem 0xd0000000-0xebffffff window] (subtractive decode) [ 1.834569] pci 0000:00:1e.0: bridge window [mem 0x380000000000-0x38007fffffff window] (subtractive decode) [ 1.834647] pci_bus 0000:00: on NUMA node 0 [ 1.835819] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 10 *11 12 14 15) [ 1.844819] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 *10 11 12 14 15) [ 1.853810] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15) [ 1.862819] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15) [ 1.871815] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 *5 6 10 11 12 14 15) [ 1.880805] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 10 *11 12 14 15) [ 1.889783] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 *10 11 12 14 15) [ 1.898781] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 *10 11 12 14 15) [ 1.907960] ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 80-fe]) [ 1.915168] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 1.925067] acpi PNP0A08:01: _OSC: platform does not support [AER] [ 1.932507] acpi PNP0A08:01: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] [ 1.941562] acpi PNP0A08:01: FADT indicates ASPM is unsupported, using BIOS configuration [ 1.951600] PCI host bridge to bus 0000:80 [ 1.956471] pci_bus 0000:80: root bus resource [bus 80-fe] [ 1.962892] pci_bus 0000:80: root bus resource [io 0x03b0-0x03df window] [ 1.970771] pci_bus 0000:80: root bus resource [io 0xc000-0xffff window] [ 1.978651] pci_bus 0000:80: root bus resource [mem 0x000a0000-0x000bffff window] [ 1.987512] pci_bus 0000:80: root bus resource [mem 0xec000000-0xfbffffff window] [ 1.996379] pci_bus 0000:80: root bus resource [mem 0x380080000000-0x3800ffffffff window] [ 2.006031] pci 0000:80:01.0: [8086:3c02] type 01 class 0x060400 [ 2.006141] pci 0000:80:01.0: PME# supported from D0 D3hot D3cold [ 2.006215] pci 0000:80:01.0: System wakeup disabled by ACPI [ 2.012930] pci 0000:80:02.0: [8086:3c04] type 01 class 0x060400 [ 2.013038] pci 0000:80:02.0: PME# supported from D0 D3hot D3cold [ 2.013102] pci 0000:80:02.0: System wakeup disabled by ACPI [ 2.019788] pci 0000:80:03.0: [8086:3c08] type 01 class 0x060400 [ 2.019895] pci 0000:80:03.0: PME# supported from D0 D3hot D3cold [ 2.019964] pci 0000:80:03.0: System wakeup disabled by ACPI [ 2.026649] pci 0000:80:03.2: [8086:3c0a] type 01 class 0x060400 [ 2.026753] pci 0000:80:03.2: PME# supported from D0 D3hot D3cold [ 2.026815] pci 0000:80:03.2: System wakeup disabled by ACPI [ 2.033501] pci 0000:80:04.0: [8086:3c20] type 00 class 0x088000 [ 2.033525] pci 0000:80:04.0: reg 0x10: [mem 0xfbf70000-0xfbf73fff 64bit] [ 2.033698] pci 0000:80:04.1: [8086:3c21] type 00 class 0x088000 [ 2.033720] pci 0000:80:04.1: reg 0x10: [mem 0xfbf60000-0xfbf63fff 64bit] [ 2.033893] pci 0000:80:04.2: [8086:3c22] type 00 class 0x088000 [ 2.033916] pci 0000:80:04.2: reg 0x10: [mem 0xfbf50000-0xfbf53fff 64bit] [ 2.034087] pci 0000:80:04.3: [8086:3c23] type 00 class 0x088000 [ 2.034110] pci 0000:80:04.3: reg 0x10: [mem 0xfbf40000-0xfbf43fff 64bit] [ 2.034278] pci 0000:80:04.4: [8086:3c24] type 00 class 0x088000 [ 2.034300] pci 0000:80:04.4: reg 0x10: [mem 0xfbf30000-0xfbf33fff 64bit] [ 2.034472] pci 0000:80:04.5: [8086:3c25] type 00 class 0x088000 [ 2.034495] pci 0000:80:04.5: reg 0x10: [mem 0xfbf20000-0xfbf23fff 64bit] [ 2.034666] pci 0000:80:04.6: [8086:3c26] type 00 class 0x088000 [ 2.034689] pci 0000:80:04.6: reg 0x10: [mem 0xfbf10000-0xfbf13fff 64bit] [ 2.034853] pci 0000:80:04.7: [8086:3c27] type 00 class 0x088000 [ 2.034875] pci 0000:80:04.7: reg 0x10: [mem 0xfbf00000-0xfbf03fff 64bit] [ 2.035042] pci 0000:80:05.0: [8086:3c28] type 00 class 0x088000 [ 2.035198] pci 0000:80:05.2: [8086:3c2a] type 00 class 0x088000 [ 2.035351] pci 0000:80:05.4: [8086:3c2c] type 00 class 0x080020 [ 2.035369] pci 0000:80:05.4: reg 0x10: [mem 0xec200000-0xec200fff] [ 2.035617] pci 0000:80:01.0: PCI bridge to [bus 81] [ 2.041557] pci 0000:82:00.0: [1000:0087] type 00 class 0x010700 [ 2.041574] pci 0000:82:00.0: reg 0x10: [io 0xc000-0xc0ff] [ 2.041589] pci 0000:82:00.0: reg 0x14: [mem 0xec140000-0xec14ffff 64bit] [ 2.041603] pci 0000:82:00.0: reg 0x1c: [mem 0xec100000-0xec13ffff 64bit] [ 2.041621] pci 0000:82:00.0: reg 0x30: [mem 0xec000000-0xec0fffff pref] [ 2.041684] pci 0000:82:00.0: supports D1 D2 [ 2.043474] pci 0000:80:02.0: PCI bridge to [bus 82] [ 2.049317] pci 0000:80:02.0: bridge window [io 0xc000-0xcfff] [ 2.049323] pci 0000:80:02.0: bridge window [mem 0xec000000-0xec1fffff] [ 2.049411] pci 0000:80:03.0: PCI bridge to [bus 83] [ 2.055333] pci 0000:80:03.2: PCI bridge to [bus 84] [ 2.061220] pci_bus 0000:80: on NUMA node 1 [ 2.071164] ACPI: PCI Root Bridge [UCR0] (domain 0000 [bus 7f]) [ 2.078078] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 2.087761] acpi PNP0A03:00: _OSC: platform does not support [PCIeHotplug PME AER] [ 2.096768] acpi PNP0A03:00: _OSC: OS now controls [PCIeCapability] [ 2.104066] acpi PNP0A03:00: FADT indicates ASPM is unsupported, using BIOS configuration [ 2.113810] PCI host bridge to bus 0000:7f [ 2.118678] pci_bus 0000:7f: root bus resource [bus 7f] [ 2.124820] pci 0000:7f:08.0: [8086:3c80] type 00 class 0x088000 [ 2.124921] pci 0000:7f:09.0: [8086:3c90] type 00 class 0x088000 [ 2.125011] pci 0000:7f:0a.0: [8086:3cc0] type 00 class 0x088000 [ 2.125090] pci 0000:7f:0a.1: [8086:3cc1] type 00 class 0x088000 [ 2.125169] pci 0000:7f:0a.2: [8086:3cc2] type 00 class 0x088000 [ 2.125248] pci 0000:7f:0a.3: [8086:3cd0] type 00 class 0x088000 [ 2.125335] pci 0000:7f:0b.0: [8086:3ce0] type 00 class 0x088000 [ 2.125414] pci 0000:7f:0b.3: [8086:3ce3] type 00 class 0x088000 [ 2.125501] pci 0000:7f:0c.0: [8086:3ce8] type 00 class 0x088000 [ 2.125577] pci 0000:7f:0c.1: [8086:3ce8] type 00 class 0x088000 [ 2.125656] pci 0000:7f:0c.2: [8086:3ce8] type 00 class 0x088000 [ 2.125734] pci 0000:7f:0c.3: [8086:3ce8] type 00 class 0x088000 [ 2.125818] pci 0000:7f:0c.6: [8086:3cf4] type 00 class 0x088000 [ 2.125897] pci 0000:7f:0c.7: [8086:3cf6] type 00 class 0x088000 [ 2.125978] pci 0000:7f:0d.0: [8086:3ce8] type 00 class 0x088000 [ 2.126056] pci 0000:7f:0d.1: [8086:3ce8] type 00 class 0x088000 [ 2.126135] pci 0000:7f:0d.2: [8086:3ce8] type 00 class 0x088000 [ 2.126212] pci 0000:7f:0d.3: [8086:3ce8] type 00 class 0x088000 [ 2.126294] pci 0000:7f:0d.6: [8086:3cf5] type 00 class 0x088000 [ 2.126375] pci 0000:7f:0e.0: [8086:3ca0] type 00 class 0x088000 [ 2.126459] pci 0000:7f:0e.1: [8086:3c46] type 00 class 0x110100 [ 2.126553] pci 0000:7f:0f.0: [8086:3ca8] type 00 class 0x088000 [ 2.126658] pci 0000:7f:0f.1: [8086:3c71] type 00 class 0x088000 [ 2.126757] pci 0000:7f:0f.2: [8086:3caa] type 00 class 0x088000 [ 2.126862] pci 0000:7f:0f.3: [8086:3cab] type 00 class 0x088000 [ 2.126962] pci 0000:7f:0f.4: [8086:3cac] type 00 class 0x088000 [ 2.127066] pci 0000:7f:0f.5: [8086:3cad] type 00 class 0x088000 [ 2.127161] pci 0000:7f:0f.6: [8086:3cae] type 00 class 0x088000 [ 2.127248] pci 0000:7f:10.0: [8086:3cb0] type 00 class 0x088000 [ 2.127351] pci 0000:7f:10.1: [8086:3cb1] type 00 class 0x088000 [ 2.127453] pci 0000:7f:10.2: [8086:3cb2] type 00 class 0x088000 [ 2.127553] pci 0000:7f:10.3: [8086:3cb3] type 00 class 0x088000 [ 2.127656] pci 0000:7f:10.4: [8086:3cb4] type 00 class 0x088000 [ 2.127757] pci 0000:7f:10.5: [8086:3cb5] type 00 class 0x088000 [ 2.127857] pci 0000:7f:10.6: [8086:3cb6] type 00 class 0x088000 [ 2.127957] pci 0000:7f:10.7: [8086:3cb7] type 00 class 0x088000 [ 2.128055] pci 0000:7f:11.0: [8086:3cb8] type 00 class 0x088000 [ 2.128149] pci 0000:7f:13.0: [8086:3ce4] type 00 class 0x088000 [ 2.128231] pci 0000:7f:13.1: [8086:3c43] type 00 class 0x110100 [ 2.128318] pci 0000:7f:13.4: [8086:3ce6] type 00 class 0x110100 [ 2.128399] pci 0000:7f:13.5: [8086:3c44] type 00 class 0x110100 [ 2.128480] pci 0000:7f:13.6: [8086:3c45] type 00 class 0x088000 [ 2.128663] ACPI: PCI Root Bridge [UCR1] (domain 0000 [bus ff]) [ 2.135579] acpi PNP0A03:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] [ 2.145271] acpi PNP0A03:01: _OSC: platform does not support [PCIeHotplug PME AER] [ 2.159002] acpi PNP0A03:01: _OSC: OS now controls [PCIeCapability] [ 2.166296] acpi PNP0A03:01: FADT indicates ASPM is unsupported, using BIOS configuration [ 2.176055] PCI host bridge to bus 0000:ff [ 2.180922] pci_bus 0000:ff: root bus resource [bus ff] [ 2.187065] pci 0000:ff:08.0: [8086:3c80] type 00 class 0x088000 [ 2.187193] pci 0000:ff:09.0: [8086:3c90] type 00 class 0x088000 [ 2.187286] pci 0000:ff:0a.0: [8086:3cc0] type 00 class 0x088000 [ 2.187373] pci 0000:ff:0a.1: [8086:3cc1] type 00 class 0x088000 [ 2.187459] pci 0000:ff:0a.2: [8086:3cc2] type 00 class 0x088000 [ 2.187543] pci 0000:ff:0a.3: [8086:3cd0] type 00 class 0x088000 [ 2.187639] pci 0000:ff:0b.0: [8086:3ce0] type 00 class 0x088000 [ 2.187720] pci 0000:ff:0b.3: [8086:3ce3] type 00 class 0x088000 [ 2.187798] pci 0000:ff:0c.0: [8086:3ce8] type 00 class 0x088000 [ 2.187877] pci 0000:ff:0c.1: [8086:3ce8] type 00 class 0x088000 [ 2.187956] pci 0000:ff:0c.2: [8086:3ce8] type 00 class 0x088000 [ 2.188036] pci 0000:ff:0c.3: [8086:3ce8] type 00 class 0x088000 [ 2.188130] pci 0000:ff:0c.6: [8086:3cf4] type 00 class 0x088000 [ 2.188213] pci 0000:ff:0c.7: [8086:3cf6] type 00 class 0x088000 [ 2.188295] pci 0000:ff:0d.0: [8086:3ce8] type 00 class 0x088000 [ 2.188376] pci 0000:ff:0d.1: [8086:3ce8] type 00 class 0x088000 [ 2.188461] pci 0000:ff:0d.2: [8086:3ce8] type 00 class 0x088000 [ 2.188551] pci 0000:ff:0d.3: [8086:3ce8] type 00 class 0x088000 [ 2.188636] pci 0000:ff:0d.6: [8086:3cf5] type 00 class 0x088000 [ 2.188720] pci 0000:ff:0e.0: [8086:3ca0] type 00 class 0x088000 [ 2.188810] pci 0000:ff:0e.1: [8086:3c46] type 00 class 0x110100 [ 2.188909] pci 0000:ff:0f.0: [8086:3ca8] type 00 class 0x088000 [ 2.189016] pci 0000:ff:0f.1: [8086:3c71] type 00 class 0x088000 [ 2.189129] pci 0000:ff:0f.2: [8086:3caa] type 00 class 0x088000 [ 2.189235] pci 0000:ff:0f.3: [8086:3cab] type 00 class 0x088000 [ 2.189339] pci 0000:ff:0f.4: [8086:3cac] type 00 class 0x088000 [ 2.189439] pci 0000:ff:0f.5: [8086:3cad] type 00 class 0x088000 [ 2.189542] pci 0000:ff:0f.6: [8086:3cae] type 00 class 0x088000 [ 2.189637] pci 0000:ff:10.0: [8086:3cb0] type 00 class 0x088000 [ 2.189743] pci 0000:ff:10.1: [8086:3cb1] type 00 class 0x088000 [ 2.189853] pci 0000:ff:10.2: [8086:3cb2] type 00 class 0x088000 [ 2.189957] pci 0000:ff:10.3: [8086:3cb3] type 00 class 0x088000 [ 2.190067] pci 0000:ff:10.4: [8086:3cb4] type 00 class 0x088000 [ 2.190173] pci 0000:ff:10.5: [8086:3cb5] type 00 class 0x088000 [ 2.190278] pci 0000:ff:10.6: [8086:3cb6] type 00 class 0x088000 [ 2.190384] pci 0000:ff:10.7: [8086:3cb7] type 00 class 0x088000 [ 2.190494] pci 0000:ff:11.0: [8086:3cb8] type 00 class 0x088000 [ 2.190592] pci 0000:ff:13.0: [8086:3ce4] type 00 class 0x088000 [ 2.190674] pci 0000:ff:13.1: [8086:3c43] type 00 class 0x110100 [ 2.190759] pci 0000:ff:13.4: [8086:3ce6] type 00 class 0x110100 [ 2.190841] pci 0000:ff:13.5: [8086:3c44] type 00 class 0x110100 [ 2.190923] pci 0000:ff:13.6: [8086:3c45] type 00 class 0x088000 [ 2.191261] ACPI: Enabled 6 GPEs in block 00 to 3F [ 2.197253] vgaarb: device added: PCI:0000:09:00.0,decodes=io+mem,owns=io+mem,locks=none [ 2.206825] vgaarb: loaded [ 2.210133] vgaarb: bridge control possible 0000:09:00.0 [ 2.216578] SCSI subsystem initialized [ 2.221121] ACPI: bus type USB registered [ 2.225914] usbcore: registered new interface driver usbfs [ 2.232349] usbcore: registered new interface driver hub [ 2.238856] usbcore: registered new device driver usb [ 2.245331] PCI: Using ACPI for IRQ routing [ 2.256892] PCI: pci_cache_line_size set to 64 bytes [ 2.257246] e820: reserve RAM buffer [mem 0x0008d400-0x0008ffff] [ 2.257250] e820: reserve RAM buffer [mem 0xbb3c7000-0xbbffffff] [ 2.257252] e820: reserve RAM buffer [mem 0xbe000000-0xbfffffff] [ 2.257651] NetLabel: Initializing [ 2.261739] NetLabel: domain hash size = 128 [ 2.266895] NetLabel: protocols = UNLABELED CIPSOv4 [ 2.272761] NetLabel: unlabeled traffic allowed by default [ 2.279351] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0 [ 2.287332] hpet0: 8 comparators, 64-bit 14.318180 MHz counter [ 2.296192] amd_nb: Cannot enumerate AMD northbridges [ 2.302319] Switched to clocksource hpet [ 2.321189] pnp: PnP ACPI init [ 2.324918] ACPI: bus type PNP registered [ 2.329932] system 00:00: [io 0x0680-0x069f] has been reserved [ 2.336850] system 00:00: [io 0xffff] has been reserved [ 2.343082] system 00:00: [io 0xffff] has been reserved [ 2.349311] system 00:00: [io 0xffff] has been reserved [ 2.355550] system 00:00: [io 0x0400-0x0453] could not be reserved [ 2.362850] system 00:00: [io 0x0458-0x047f] has been reserved [ 2.369760] system 00:00: [io 0x0500-0x057f] has been reserved [ 2.376670] system 00:00: [io 0x0600-0x061f] has been reserved [ 2.383582] system 00:00: [io 0x0ca2-0x0ca5] has been reserved [ 2.390482] system 00:00: [io 0x0cf9] could not be reserved [ 2.397101] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active) [ 2.397156] pnp 00:01: Plug and Play ACPI device, IDs PNP0b00 (active) [ 2.397277] system 00:02: [io 0x0454-0x0457] has been reserved [ 2.404196] system 00:02: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active) [ 2.404575] pnp 00:03: [dma 0 disabled] [ 2.404685] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) [ 2.404941] pnp 00:04: [dma 0 disabled] [ 2.405042] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) [ 2.405607] system 00:05: [mem 0xfed1c000-0xfed1ffff] has been reserved [ 2.413305] system 00:05: [mem 0xebfff000-0xebffffff] has been reserved [ 2.420986] system 00:05: [mem 0xc0000000-0xcfffffff] has been reserved [ 2.428675] system 00:05: [mem 0xfed20000-0xfed3ffff] has been reserved [ 2.436362] system 00:05: [mem 0xfed45000-0xfed8ffff] has been reserved [ 2.444054] system 00:05: [mem 0xff000000-0xffffffff] could not be reserved [ 2.452133] system 00:05: [mem 0xfee00000-0xfeefffff] could not be reserved [ 2.460209] system 00:05: [mem 0xfec00000-0xfecfffff] could not be reserved [ 2.468277] system 00:05: [mem 0xd0f70000-0xd0f70fff] has been reserved [ 2.475966] system 00:05: Plug and Play ACPI device, IDs PNP0c02 (active) [ 2.476462] system 00:06: [mem 0x00000000-0x0009cfff] could not be reserved [ 2.484547] system 00:06: Plug and Play ACPI device, IDs PNP0c01 (active) [ 2.485032] pnp: PnP ACPI: found 7 devices [ 2.489903] ACPI: bus type PNP unregistered [ 2.503408] pci 0000:00:01.0: PCI bridge to [bus 01] [ 2.509266] pci 0000:00:01.1: PCI bridge to [bus 02-03] [ 2.515401] pci 0000:00:01.1: bridge window [io 0x3000-0x3fff] [ 2.522507] pci 0000:00:01.1: bridge window [mem 0xd0b00000-0xd0cfffff] [ 2.530398] pci 0000:00:02.0: PCI bridge to [bus 04] [ 2.536243] pci 0000:00:02.0: bridge window [mem 0xd0e00000-0xd0efffff] [ 2.544121] pci 0000:00:02.0: bridge window [mem 0xeb000000-0xeb7fffff 64bit pref] [ 2.553281] pci 0000:00:02.2: PCI bridge to [bus 05] [ 2.559132] pci 0000:00:03.0: PCI bridge to [bus 06] [ 2.564975] pci 0000:00:03.0: bridge window [io 0x2000-0x2fff] [ 2.572081] pci 0000:00:03.0: bridge window [mem 0xd0900000-0xd0afffff] [ 2.579969] pci 0000:00:11.0: PCI bridge to [bus 07] [ 2.585813] pci 0000:00:11.0: bridge window [io 0x1000-0x1fff] [ 2.592923] pci 0000:00:11.0: bridge window [mem 0xd0d00000-0xd0dfffff] [ 2.600805] pci 0000:00:11.0: bridge window [mem 0xeb800000-0xebcfffff 64bit pref] [ 2.609967] pci 0000:00:1c.0: PCI bridge to [bus 08] [ 2.615824] pci 0000:00:1c.7: PCI bridge to [bus 09] [ 2.621667] pci 0000:00:1c.7: bridge window [mem 0xd0000000-0xd08fffff] [ 2.629553] pci 0000:00:1c.7: bridge window [mem 0xea000000-0xeaffffff 64bit pref] [ 2.638713] pci 0000:00:1e.0: PCI bridge to [bus 0a] [ 2.644567] pci_bus 0000:00: resource 4 [io 0x0000-0xbfff window] [ 2.644571] pci_bus 0000:00: resource 5 [mem 0x000a0000-0x000bffff window] [ 2.644574] pci_bus 0000:00: resource 6 [mem 0x000c0000-0x000c3fff window] [ 2.644577] pci_bus 0000:00: resource 7 [mem 0x000c4000-0x000c7fff window] [ 2.644580] pci_bus 0000:00: resource 8 [mem 0x000c8000-0x000cbfff window] [ 2.644584] pci_bus 0000:00: resource 9 [mem 0x000cc000-0x000cffff window] [ 2.644587] pci_bus 0000:00: resource 10 [mem 0x000d4000-0x000d7fff window] [ 2.644590] pci_bus 0000:00: resource 11 [mem 0x000dc000-0x000dffff window] [ 2.644594] pci_bus 0000:00: resource 12 [mem 0x000e0000-0x000e3fff window] [ 2.644597] pci_bus 0000:00: resource 13 [mem 0x000e4000-0x000e7fff window] [ 2.644600] pci_bus 0000:00: resource 14 [mem 0x000e8000-0x000ebfff window] [ 2.644604] pci_bus 0000:00: resource 15 [mem 0x000ec000-0x000effff window] [ 2.644607] pci_bus 0000:00: resource 16 [mem 0x000f0000-0x000fffff window] [ 2.644610] pci_bus 0000:00: resource 17 [mem 0xd0000000-0xebffffff window] [ 2.644614] pci_bus 0000:00: resource 18 [mem 0x380000000000-0x38007fffffff window] [ 2.644618] pci_bus 0000:02: resource 0 [io 0x3000-0x3fff] [ 2.644621] pci_bus 0000:02: resource 1 [mem 0xd0b00000-0xd0cfffff] [ 2.644625] pci_bus 0000:04: resource 1 [mem 0xd0e00000-0xd0efffff] [ 2.644628] pci_bus 0000:04: resource 2 [mem 0xeb000000-0xeb7fffff 64bit pref] [ 2.644631] pci_bus 0000:06: resource 0 [io 0x2000-0x2fff] [ 2.644634] pci_bus 0000:06: resource 1 [mem 0xd0900000-0xd0afffff] [ 2.644638] pci_bus 0000:07: resource 0 [io 0x1000-0x1fff] [ 2.644641] pci_bus 0000:07: resource 1 [mem 0xd0d00000-0xd0dfffff] [ 2.644644] pci_bus 0000:07: resource 2 [mem 0xeb800000-0xebcfffff 64bit pref] [ 2.644648] pci_bus 0000:09: resource 1 [mem 0xd0000000-0xd08fffff] [ 2.644652] pci_bus 0000:09: resource 2 [mem 0xea000000-0xeaffffff 64bit pref] [ 2.644655] pci_bus 0000:0a: resource 4 [io 0x0000-0xbfff window] [ 2.644659] pci_bus 0000:0a: resource 5 [mem 0x000a0000-0x000bffff window] [ 2.644662] pci_bus 0000:0a: resource 6 [mem 0x000c0000-0x000c3fff window] [ 2.644665] pci_bus 0000:0a: resource 7 [mem 0x000c4000-0x000c7fff window] [ 2.644668] pci_bus 0000:0a: resource 8 [mem 0x000c8000-0x000cbfff window] [ 2.644672] pci_bus 0000:0a: resource 9 [mem 0x000cc000-0x000cffff window] [ 2.644675] pci_bus 0000:0a: resource 10 [mem 0x000d4000-0x000d7fff window] [ 2.644678] pci_bus 0000:0a: resource 11 [mem 0x000dc000-0x000dffff window] [ 2.644681] pci_bus 0000:0a: resource 12 [mem 0x000e0000-0x000e3fff window] [ 2.644685] pci_bus 0000:0a: resource 13 [mem 0x000e4000-0x000e7fff window] [ 2.644688] pci_bus 0000:0a: resource 14 [mem 0x000e8000-0x000ebfff window] [ 2.644691] pci_bus 0000:0a: resource 15 [mem 0x000ec000-0x000effff window] [ 2.644694] pci_bus 0000:0a: resource 16 [mem 0x000f0000-0x000fffff window] [ 2.644698] pci_bus 0000:0a: resource 17 [mem 0xd0000000-0xebffffff window] [ 2.644701] pci_bus 0000:0a: resource 18 [mem 0x380000000000-0x38007fffffff window] [ 2.644740] pci 0000:80:01.0: PCI bridge to [bus 81] [ 2.650591] pci 0000:80:02.0: PCI bridge to [bus 82] [ 2.656433] pci 0000:80:02.0: bridge window [io 0xc000-0xcfff] [ 2.663542] pci 0000:80:02.0: bridge window [mem 0xec000000-0xec1fffff] [ 2.671438] pci 0000:80:03.0: PCI bridge to [bus 83] [ 2.677287] pci 0000:80:03.2: PCI bridge to [bus 84] [ 2.683137] pci_bus 0000:80: resource 4 [io 0x03b0-0x03df window] [ 2.683140] pci_bus 0000:80: resource 5 [io 0xc000-0xffff window] [ 2.683143] pci_bus 0000:80: resource 6 [mem 0x000a0000-0x000bffff window] [ 2.683147] pci_bus 0000:80: resource 7 [mem 0xec000000-0xfbffffff window] [ 2.683150] pci_bus 0000:80: resource 8 [mem 0x380080000000-0x3800ffffffff window] [ 2.683154] pci_bus 0000:82: resource 0 [io 0xc000-0xcfff] [ 2.683157] pci_bus 0000:82: resource 1 [mem 0xec000000-0xec1fffff] [ 2.683506] NET: Registered protocol family 2 [ 2.690880] TCP established hash table entries: 262144 (order: 9, 2097152 bytes) [ 2.700539] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) [ 2.708573] TCP: Hash tables configured (established 262144 bind 65536) [ 2.716361] TCP: reno registered [ 2.720428] UDP hash table entries: 16384 (order: 7, 524288 bytes) [ 2.727880] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes) [ 2.736920] NET: Registered protocol family 1 [ 3.842307] pci 0000:00:1a.0: EHCI: BIOS handoff failed (BIOS bug?) 01010001 [ 4.950308] pci 0000:00:1d.0: EHCI: BIOS handoff failed (BIOS bug?) 01010001 [ 4.958768] PCI: CLS mismatch (64 != 32), using 64 bytes [ 4.958792] pci 0000:09:00.0: Boot video device [ 4.959082] Unpacking initramfs... [ 5.806617] Freeing initrd memory: 27200k freed [ 5.822590] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 5.830101] software IO TLB [mem 0xb73c7000-0xbb3c7000] (64MB) mapped at [ffff8800b73c7000-ffff8800bb3c6fff] [ 5.842050] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer [ 5.851425] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules [ 5.858154] RAPL PMU: hw unit of domain package 2^-16 Joules [ 5.864778] RAPL PMU: hw unit of domain dram 2^-16 Joules [ 5.882902] sha1_ssse3: Using AVX optimized SHA-1 implementation [ 5.890061] sha256_ssse3: Using AVX optimized SHA-256 implementation [ 5.902558] futex hash table entries: 65536 (order: 10, 4194304 bytes) [ 5.911113] Initialise system trusted keyring [ 5.916424] audit: initializing netlink socket (disabled) [ 5.922799] type=2000 audit(1510797680.526:1): initialized [ 5.980159] HugeTLB registered 1 GB page size, pre-allocated 0 pages [ 5.987557] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 5.997101] zpool: loaded [ 6.000334] zbud: loaded [ 6.004406] VFS: Disk quotas dquot_6.6.0 [ 6.009599] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 6.018508] msgmni has been set to 32768 [ 6.023589] Key type big_key registered [ 6.028184] SELinux: Registering netfilter hooks [ 6.030669] NET: Registered protocol family 38 [ 6.035958] Key type asymmetric registered [ 6.040837] Asymmetric key parser 'x509' registered [ 6.046737] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250) [ 6.055783] io scheduler noop registered [ 6.060476] io scheduler deadline registered (default) [ 6.066777] io scheduler cfq registered [ 6.072339] pcieport 0000:00:01.0: irq 25 for MSI/MSI-X [ 6.072709] pcieport 0000:00:01.1: irq 26 for MSI/MSI-X [ 6.073080] pcieport 0000:00:02.0: irq 27 for MSI/MSI-X [ 6.073443] pcieport 0000:00:02.2: irq 28 for MSI/MSI-X [ 6.073885] pcieport 0000:00:03.0: irq 29 for MSI/MSI-X [ 6.074265] pcieport 0000:00:11.0: irq 30 for MSI/MSI-X [ 6.074636] pcieport 0000:00:1c.0: irq 31 for MSI/MSI-X [ 6.075093] pcieport 0000:00:1c.7: irq 32 for MSI/MSI-X [ 6.075650] pcieport 0000:80:01.0: irq 34 for MSI/MSI-X [ 6.075935] pcieport 0000:80:02.0: irq 35 for MSI/MSI-X [ 6.076131] pcieport 0000:80:03.0: irq 36 for MSI/MSI-X [ 6.076335] pcieport 0000:80:03.2: irq 37 for MSI/MSI-X [ 6.076488] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt [ 6.084589] pcie_pme 0000:00:01.0:pcie01: service driver pcie_pme loaded [ 6.084639] pcieport 0000:00:01.1: Signaling PME through PCIe PME interrupt [ 6.092730] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt [ 6.100332] pci 0000:02:00.1: Signaling PME through PCIe PME interrupt [ 6.107938] pci 0000:02:00.2: Signaling PME through PCIe PME interrupt [ 6.115537] pci 0000:02:00.3: Signaling PME through PCIe PME interrupt [ 6.123152] pcie_pme 0000:00:01.1:pcie01: service driver pcie_pme loaded [ 6.123186] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt [ 6.131268] pci 0000:04:00.0: Signaling PME through PCIe PME interrupt [ 6.138876] pcie_pme 0000:00:02.0:pcie01: service driver pcie_pme loaded [ 6.138910] pcieport 0000:00:02.2: Signaling PME through PCIe PME interrupt [ 6.147016] pcie_pme 0000:00:02.2:pcie01: service driver pcie_pme loaded [ 6.147067] pcieport 0000:00:03.0: Signaling PME through PCIe PME interrupt [ 6.155159] pci 0000:06:00.0: Signaling PME through PCIe PME interrupt [ 6.162761] pcie_pme 0000:00:03.0:pcie01: service driver pcie_pme loaded [ 6.162790] pcieport 0000:00:11.0: Signaling PME through PCIe PME interrupt [ 6.170887] pci 0000:07:00.0: Signaling PME through PCIe PME interrupt [ 6.178495] pci 0000:07:00.3: Signaling PME through PCIe PME interrupt [ 6.186112] pcie_pme 0000:00:11.0:pcie01: service driver pcie_pme loaded [ 6.186143] pcieport 0000:00:1c.0: Signaling PME through PCIe PME interrupt [ 6.194245] pcie_pme 0000:00:1c.0:pcie01: service driver pcie_pme loaded [ 6.194279] pcieport 0000:00:1c.7: Signaling PME through PCIe PME interrupt [ 6.202369] pci 0000:09:00.0: Signaling PME through PCIe PME interrupt [ 6.209978] pcie_pme 0000:00:1c.7:pcie01: service driver pcie_pme loaded [ 6.210019] pcieport 0000:80:01.0: Signaling PME through PCIe PME interrupt [ 6.218115] pcie_pme 0000:80:01.0:pcie01: service driver pcie_pme loaded [ 6.218145] pcieport 0000:80:02.0: Signaling PME through PCIe PME interrupt [ 6.226232] pci 0000:82:00.0: Signaling PME through PCIe PME interrupt [ 6.233845] pcie_pme 0000:80:02.0:pcie01: service driver pcie_pme loaded [ 6.233885] pcieport 0000:80:03.0: Signaling PME through PCIe PME interrupt [ 6.241972] pcie_pme 0000:80:03.0:pcie01: service driver pcie_pme loaded [ 6.242002] pcieport 0000:80:03.2: Signaling PME through PCIe PME interrupt [ 6.250105] pcie_pme 0000:80:03.2:pcie01: service driver pcie_pme loaded [ 6.250201] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 [ 6.256751] pciehp: PCI Express Hot Plug Controller Driver version: 0.4 [ 6.264619] intel_idle: MWAIT substates: 0x21120 [ 6.264622] intel_idle: v0.4.1 model 0x2D [ 6.264892] intel_idle: lapic_timer_reliable_states 0xffffffff [ 6.265017] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 6.273789] ACPI: Power Button [PWRF] [ 6.278212] ACPI: Requesting acpi_cpufreq [ 6.318553] ERST: Error Record Serialization Table (ERST) support is initialized. [ 6.327445] pstore: Registered erst as persistent store backend [ 6.334535] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC. [ 6.343389] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 6.371431] 00:03: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 6.398690] 00:04: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A [ 6.405670] Non-volatile memory driver v1.3 [ 6.410678] Linux agpgart interface v0.103 [ 6.415720] crash memory driver: version 1.1 [ 6.420975] rdac: device handler registered [ 6.425993] hp_sw: device handler registered [ 6.431074] emc: device handler registered [ 6.435959] alua: device handler registered [ 6.440957] libphy: Fixed MDIO Bus: probed [ 6.445872] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 6.453482] ehci-pci: EHCI PCI platform driver [ 6.458982] ehci-pci 0000:00:1a.0: EHCI Host Controller [ 6.465170] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1 [ 6.473950] ehci-pci 0000:00:1a.0: debug port 2 [ 6.483215] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported [ 6.483255] ehci-pci 0000:00:1a.0: irq 22, io mem 0xd0f20000 [ 6.495314] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00 [ 6.502075] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 [ 6.509964] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 6.518539] usb usb1: Product: EHCI Host Controller [ 6.524287] usb usb1: Manufacturer: Linux 3.10.0-693.5.2.el7_lustre.x86_64 ehci_hcd [ 6.533326] usb usb1: SerialNumber: 0000:00:1a.0 [ 6.543454] hub 1-0:1.0: USB hub found [ 6.547947] hub 1-0:1.0: 2 ports detected [ 6.552978] ehci-pci 0000:00:1d.0: EHCI Host Controller [ 6.559150] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2 [ 6.567931] ehci-pci 0000:00:1d.0: debug port 2 [ 6.577197] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported [ 6.577211] ehci-pci 0000:00:1d.0: irq 20, io mem 0xd0f10000 [ 6.589305] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00 [ 6.596050] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002 [ 6.603918] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 6.612490] usb usb2: Product: EHCI Host Controller [ 6.618232] usb usb2: Manufacturer: Linux 3.10.0-693.5.2.el7_lustre.x86_64 ehci_hcd [ 6.627283] usb usb2: SerialNumber: 0000:00:1d.0 [ 6.632807] hub 2-0:1.0: USB hub found [ 6.637281] hub 2-0:1.0: 2 ports detected [ 6.642143] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 6.649354] ohci-pci: OHCI PCI platform driver [ 6.654628] uhci_hcd: USB Universal Host Controller Interface driver [ 6.662097] usbcore: registered new interface driver usbserial [ 6.668930] usbcore: registered new interface driver usbserial_generic [ 6.676541] usbserial: USB Serial support registered for generic [ 6.683570] i8042: PNP: No PS/2 controller found. Probing ports directly. [ 6.854312] usb 1-1: new high-speed USB device number 2 using ehci-pci [ 6.879317] tsc: Refined TSC clocksource calibration: 2693.475 MHz [ 7.730470] i8042: No controller found [ 7.735002] Switched to clocksource tsc [ 7.735020] mousedev: PS/2 mouse device common for all mice [ 7.735174] rtc_cmos 00:01: RTC can wake from S4 [ 7.735422] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0 [ 7.735481] rtc_cmos 00:01: alarms up to one month, y3k, 242 bytes nvram, hpet irqs [ 7.735533] Intel P-state driver initializing. [ 7.741474] cpuidle: using governor menu [ 7.742441] hidraw: raw HID events driver (C) Jiri Kosina [ 7.742604] usbcore: registered new interface driver usbhid [ 7.742605] usbhid: USB HID core driver [ 7.742739] drop_monitor: Initializing network drop monitor service [ 7.743429] TCP: cubic registered [ 7.743441] Initializing XFRM netlink socket [ 7.744087] NET: Registered protocol family 10 [ 7.745735] NET: Registered protocol family 17 [ 7.747070] microcode: CPU0 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751558] microcode: CPU1 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751570] microcode: CPU2 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751581] microcode: CPU3 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751592] microcode: CPU4 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751608] microcode: CPU5 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751620] microcode: CPU6 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751631] microcode: CPU7 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751638] microcode: CPU8 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751648] microcode: CPU9 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751659] microcode: CPU10 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751669] microcode: CPU11 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751679] microcode: CPU12 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751689] microcode: CPU13 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751699] microcode: CPU14 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751707] microcode: CPU15 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751717] microcode: CPU16 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751734] microcode: CPU17 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751744] microcode: CPU18 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751753] microcode: CPU19 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751763] microcode: CPU20 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751773] microcode: CPU21 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751783] microcode: CPU22 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751793] microcode: CPU23 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751803] microcode: CPU24 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751813] microcode: CPU25 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751821] microcode: CPU26 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751829] microcode: CPU27 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751837] microcode: CPU28 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751850] microcode: CPU29 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751858] microcode: CPU30 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751865] microcode: CPU31 sig=0x206d7, pf=0x1, revision=0x710 [ 7.751916] microcode: Microcode Update Driver: v2.01 , Peter Oruba [ 7.752103] PM: Hibernation image not present or could not be loaded. [ 7.752107] Loading compiled-in X.509 certificates [ 7.752152] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717' [ 7.752174] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b' [ 7.753227] Loaded X.509 cert 'CentOS Linux kernel signing key: b47a4e68522da148dcade840321620b487b1ebac' [ 7.753260] registered taskstats version 1 [ 7.757274] Key type trusted registered [ 7.761071] Key type encrypted registered [ 7.764991] IMA: No TPM chip found, activating TPM-bypass! [ 7.766610] Magic number: 9:403:5 [ 7.767724] rtc_cmos 00:01: setting system clock to 2017-11-16 02:01:26 UTC (1510797686) [ 7.786348] usb 2-1: new high-speed USB device number 2 using ehci-pci [ 7.799717] usb 1-1: New USB device found, idVendor=8087, idProduct=0024 [ 7.799731] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 7.800087] hub 1-1:1.0: USB hub found [ 7.800170] hub 1-1:1.0: 6 ports detected [ 7.901674] usb 2-1: New USB device found, idVendor=8087, idProduct=0024 [ 7.901676] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 7.901967] hub 2-1:1.0: USB hub found [ 7.902045] hub 2-1:1.0: 8 ports detected [ 8.177347] usb 2-1.4: new full-speed USB device number 3 using ehci-pci [ 8.198200] Freeing unused kernel memory: 1764k freed [ 8.210744] random: systemd: uninitialized urandom read (16 bytes read) [ 8.221439] random: systemd: uninitialized urandom read (16 bytes read) [ 8.229194] random: systemd: uninitialized urandom read (16 bytes read) [ 8.239383] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) [ 8.261103] systemd[1]: Detected architecture x86-64. [ 8.267064] systemd[1]: Running in initial RAM disk. [ 8.280758] usb 2-1.4: New USB device found, idVendor=046b, idProduct=ff10 [ 8.289155] usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 8.297541] systemd[1]: Set hostname to . [ 8.305615] usb 2-1.4: Product: Virtual Keyboard and Mouse [ 8.305617] usb 2-1.4: Manufacturer: American Megatrends Inc. [ 8.305618] usb 2-1.4: SerialNumber: serial [ 8.307196] input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.0/input/input1 [ 8.357621] hid-generic 0003:046B:FF10.0001: input,hidraw0: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1d.0-1.4/input0 [ 8.376347] input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.1/input/input2 [ 8.389958] random: systemd: uninitialized urandom read (16 bytes read) [ 8.389994] random: systemd: uninitialized urandom read (16 bytes read) [ 8.390054] random: systemd: uninitialized urandom read (16 bytes read) [ 8.390159] random: systemd: uninitialized urandom read (16 bytes read) [ 8.391229] random: systemd: uninitialized urandom read (16 bytes read) [ 8.391474] random: systemd: uninitialized urandom read (16 bytes read) [ 8.391788] random: systemd: uninitialized urandom read (16 bytes read) [ 8.414811] hid-generic 0003:046B:FF10.0002: input,hidraw1: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:00:1d.0-1.4/input1 [ 8.467483] systemd[1]: Reached target Swap. [ 8.472622] systemd[1]: Starting Swap. [ 8.482425] systemd[1]: Reached target Timers. [ 8.487732] systemd[1]: Starting Timers. [ 8.497439] systemd[1]: Created slice Root Slice. [ 8.503022] systemd[1]: Starting Root Slice. [ 8.513414] systemd[1]: Listening on Journal Socket. [ 8.519298] systemd[1]: Starting Journal Socket. [ 8.531452] systemd[1]: Listening on udev Kernel Socket. [ 8.537724] systemd[1]: Starting udev Kernel Socket. [ 8.550447] systemd[1]: Listening on udev Control Socket. [ 8.556803] systemd[1]: Starting udev Control Socket. [ 8.567423] systemd[1]: Reached target Sockets. [ 8.572803] systemd[1]: Starting Sockets. [ 8.584430] systemd[1]: Created slice System Slice. [ 8.590210] systemd[1]: Starting System Slice. [ 8.596233] systemd[1]: Starting Journal Service... [ 8.608183] systemd[1]: Starting dracut cmdline hook... [ 8.620350] systemd[1]: Starting Load Kernel Modules... [ 8.632158] systemd[1]: Starting Create list of required static device nodes for the current kernel... [ 8.657362] systemd[1]: Reached target Slices. [ 8.662669] systemd[1]: Starting Slices. [ 8.667895] systemd[1]: Starting Setup Virtual Console... [ 8.694358] systemd[1]: Reached target Local File Systems. [ 8.700792] systemd[1]: Starting Local File Systems. [ 8.712358] systemd[1]: Started Journal Service. [ 8.894651] mlx_compat: loading out-of-tree module taints kernel. [ 8.903637] mlx_compat: module verification failed: signature and/or required key missing - tainting kernel [ 8.917020] Compat-mlnx-ofed backport release: f36c870 [ 8.924455] Backport based on mlnx_ofed/mlnx-ofa_kernel-4.0.git f36c870 [ 8.932715] compat.git: mlnx_ofed/mlnx-ofa_kernel-4.0.git [ 8.941633] libata version 3.00 loaded. [ 8.942278] dca service started, version 1.12.1 [ 8.949198] [drm] Initialized [ 8.949248] mlx4_core: Mellanox ConnectX core driver v4.2-1.0.0 [ 8.949277] mlx4_core: Initializing 0000:04:00.0 [ 8.966225] pps_core: LinuxPPS API ver. 1 registered [ 8.966226] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 8.967269] mpt2sas version 20.103.00.00 loaded [ 8.967334] mpt2sas 0000:06:00.0: can't disable ASPM; OS doesn't have ASPM control [ 8.967515] mpt2sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (32511224 kB) [ 8.968427] PTP clock support registered [ 9.019987] ahci 0000:00:1f.2: version 3.0 [ 9.020267] ahci 0000:00:1f.2: irq 40 for MSI/MSI-X [ 9.021663] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k [ 9.021664] isci: Intel(R) C600 SAS Controller Driver - version 1.2.0 [ 9.021665] igb: Copyright (c) 2007-2014 Intel Corporation. [ 9.021804] mpt2sas_cm0: MSI-X vectors supported: 16, no of cores: 32, max_msix_vectors: -1 [ 9.021897] mpt2sas 0000:06:00.0: irq 42 for MSI/MSI-X [ 9.021959] mpt2sas 0000:06:00.0: irq 43 for MSI/MSI-X [ 9.022019] mpt2sas 0000:06:00.0: irq 44 for MSI/MSI-X [ 9.022081] mpt2sas 0000:06:00.0: irq 45 for MSI/MSI-X [ 9.022112] igb 0000:02:00.0: irq 46 for MSI/MSI-X [ 9.022138] mpt2sas 0000:06:00.0: irq 47 for MSI/MSI-X [ 9.022198] igb 0000:02:00.0: irq 48 for MSI/MSI-X [ 9.022199] mpt2sas 0000:06:00.0: irq 46 for MSI/MSI-X [ 9.022260] igb 0000:02:00.0: irq 49 for MSI/MSI-X [ 9.022261] mpt2sas 0000:06:00.0: irq 50 for MSI/MSI-X [ 9.022326] mpt2sas 0000:06:00.0: irq 52 for MSI/MSI-X [ 9.022327] igb 0000:02:00.0: irq 51 for MSI/MSI-X [ 9.022391] mpt2sas 0000:06:00.0: irq 53 for MSI/MSI-X [ 9.022392] igb 0000:02:00.0: irq 54 for MSI/MSI-X [ 9.022453] mpt2sas 0000:06:00.0: irq 55 for MSI/MSI-X [ 9.022454] igb 0000:02:00.0: irq 56 for MSI/MSI-X [ 9.022520] mpt2sas 0000:06:00.0: irq 57 for MSI/MSI-X [ 9.022520] igb 0000:02:00.0: irq 58 for MSI/MSI-X [ 9.022581] mpt2sas 0000:06:00.0: irq 59 for MSI/MSI-X [ 9.022582] igb 0000:02:00.0: irq 60 for MSI/MSI-X [ 9.022644] mpt2sas 0000:06:00.0: irq 61 for MSI/MSI-X [ 9.022644] igb 0000:02:00.0: irq 62 for MSI/MSI-X [ 9.022705] mpt2sas 0000:06:00.0: irq 63 for MSI/MSI-X [ 9.022706] igb 0000:02:00.0: irq 64 for MSI/MSI-X [ 9.022734] igb 0000:02:00.0: PHY reset is blocked due to SOL/IDER session. [ 9.022746] isci 0000:07:00.0: driver configured for rev: 6 silicon [ 9.022749] isci 0000:07:00.0: OEM parameter table found in OROM [ 9.022751] isci 0000:07:00.0: OEM SAS parameters (version: 1.1) loaded (platform) [ 9.022768] mpt2sas 0000:06:00.0: irq 65 for MSI/MSI-X [ 9.022830] mpt2sas 0000:06:00.0: irq 66 for MSI/MSI-X [ 9.022896] isci 0000:07:00.0: SCU controller 0: phy 3-0 cables: {short, short, short, short} [ 9.022941] mpt2sas0-msix0: PCI-MSI-X enabled: IRQ 42 [ 9.022942] mpt2sas0-msix1: PCI-MSI-X enabled: IRQ 43 [ 9.022942] mpt2sas0-msix2: PCI-MSI-X enabled: IRQ 44 [ 9.022943] mpt2sas0-msix3: PCI-MSI-X enabled: IRQ 45 [ 9.022943] mpt2sas0-msix4: PCI-MSI-X enabled: IRQ 47 [ 9.022944] mpt2sas0-msix5: PCI-MSI-X enabled: IRQ 46 [ 9.022944] mpt2sas0-msix6: PCI-MSI-X enabled: IRQ 50 [ 9.022945] mpt2sas0-msix7: PCI-MSI-X enabled: IRQ 52 [ 9.022945] mpt2sas0-msix8: PCI-MSI-X enabled: IRQ 53 [ 9.022945] mpt2sas0-msix9: PCI-MSI-X enabled: IRQ 55 [ 9.022946] mpt2sas0-msix10: PCI-MSI-X enabled: IRQ 57 [ 9.022946] mpt2sas0-msix11: PCI-MSI-X enabled: IRQ 59 [ 9.022947] mpt2sas0-msix12: PCI-MSI-X enabled: IRQ 61 [ 9.022947] mpt2sas0-msix13: PCI-MSI-X enabled: IRQ 63 [ 9.022948] mpt2sas0-msix14: PCI-MSI-X enabled: IRQ 65 [ 9.022948] mpt2sas0-msix15: PCI-MSI-X enabled: IRQ 66 [ 9.022949] mpt2sas_cm0: iomem(0x00000000d0a40000), mapped(0xffffc90003b20000), size(65536) [ 9.022950] mpt2sas_cm0: ioport(0x0000000000002000), size(256) [ 9.025358] scsi host1: isci [ 9.025914] isci 0000:07:00.0: irq 67 for MSI/MSI-X [ 9.025976] isci 0000:07:00.0: irq 68 for MSI/MSI-X [ 9.031407] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode [ 9.031409] ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst [ 9.041823] scsi host2: ahci [ 9.041969] scsi host3: ahci [ 9.042045] scsi host4: ahci [ 9.042116] scsi host5: ahci [ 9.042188] scsi host6: ahci [ 9.042254] scsi host7: ahci [ 9.042291] ata1: SATA max UDMA/133 abar m2048@0xd0f00000 port 0xd0f00100 irq 40 [ 9.042292] ata2: SATA max UDMA/133 abar m2048@0xd0f00000 port 0xd0f00180 irq 40 [ 9.042295] ata3: SATA max UDMA/133 abar m2048@0xd0f00000 port 0xd0f00200 irq 40 [ 9.042297] ata4: SATA max UDMA/133 abar m2048@0xd0f00000 port 0xd0f00280 irq 40 [ 9.042300] ata5: SATA max UDMA/133 abar m2048@0xd0f00000 port 0xd0f00300 irq 40 [ 9.042303] ata6: SATA max UDMA/133 abar m2048@0xd0f00000 port 0xd0f00380 irq 40 [ 9.067900] igb 0000:02:00.0: added PHC on eth0 [ 9.067901] igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection [ 9.067902] igb 0000:02:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 00:1e:67:6b:23:29 [ 9.067975] igb 0000:02:00.0: eth0: PBA No: 100000-000 [ 9.067976] igb 0000:02:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 9.068420] igb 0000:02:00.1: irq 70 for MSI/MSI-X [ 9.068509] igb 0000:02:00.1: irq 70 for MSI/MSI-X [ 9.068570] igb 0000:02:00.1: irq 71 for MSI/MSI-X [ 9.068635] igb 0000:02:00.1: irq 72 for MSI/MSI-X [ 9.068699] igb 0000:02:00.1: irq 73 for MSI/MSI-X [ 9.068760] igb 0000:02:00.1: irq 74 for MSI/MSI-X [ 9.068820] igb 0000:02:00.1: irq 75 for MSI/MSI-X [ 9.068881] igb 0000:02:00.1: irq 76 for MSI/MSI-X [ 9.068942] igb 0000:02:00.1: irq 77 for MSI/MSI-X [ 9.069003] igb 0000:02:00.1: irq 78 for MSI/MSI-X [ 9.069032] igb 0000:02:00.1: PHY reset is blocked due to SOL/IDER session. [ 9.077470] mpt2sas_cm0: sending message unit reset !! [ 9.078981] mpt2sas_cm0: message unit reset: SUCCESS [ 9.105724] igb 0000:02:00.1: added PHC on eth1 [ 9.105725] igb 0000:02:00.1: Intel(R) Gigabit Ethernet Network Connection [ 9.105726] igb 0000:02:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 00:1e:67:6b:23:2a [ 9.105799] igb 0000:02:00.1: eth1: PBA No: 100000-000 [ 9.105800] igb 0000:02:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 9.120629] igb 0000:02:00.2: irq 80 for MSI/MSI-X [ 9.120719] igb 0000:02:00.2: irq 80 for MSI/MSI-X [ 9.120782] igb 0000:02:00.2: irq 81 for MSI/MSI-X [ 9.120843] igb 0000:02:00.2: irq 82 for MSI/MSI-X [ 9.120904] igb 0000:02:00.2: irq 83 for MSI/MSI-X [ 9.120972] igb 0000:02:00.2: irq 84 for MSI/MSI-X [ 9.121033] igb 0000:02:00.2: irq 85 for MSI/MSI-X [ 9.121094] igb 0000:02:00.2: irq 86 for MSI/MSI-X [ 9.121156] igb 0000:02:00.2: irq 87 for MSI/MSI-X [ 9.121220] igb 0000:02:00.2: irq 88 for MSI/MSI-X [ 9.139392] mpt2sas_cm0: Allocated physical memory: size(18690 kB) [ 9.139393] mpt2sas_cm0: Current Controller Queue Depth(10104),Max Controller Queue Depth(10240) [ 9.139393] mpt2sas_cm0: Scatter Gather Elements per IO(128) [ 9.176523] igb 0000:02:00.2: added PHC on eth2 [ 9.176525] igb 0000:02:00.2: Intel(R) Gigabit Ethernet Network Connection [ 9.176526] igb 0000:02:00.2: eth2: (PCIe:5.0Gb/s:Width x4) 00:1e:67:6b:23:2b [ 9.176599] igb 0000:02:00.2: eth2: PBA No: 100000-000 [ 9.176601] igb 0000:02:00.2: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 9.183688] random: fast init done [ 9.183894] mpt2sas_cm0: LSISAS2308: FWVersion(20.00.07.00), ChipRevision(0x05), BiosVersion(07.39.00.00) [ 9.183896] mpt2sas_cm0: Protocol=( [ 9.183896] Initiator [ 9.183897] ,Target [ 9.183897] ), [ 9.183897] Capabilities=( [ 9.183898] TLR [ 9.183899] ,EEDP [ 9.183899] ,Snapshot Buffer [ 9.183899] ,Diag Trace Buffer [ 9.183900] ,Task Set Full [ 9.183900] ,NCQ [ 9.183900] ) [ 9.183948] scsi host0: Fusion MPT SAS Host [ 9.184387] mpt2sas 0000:82:00.0: can't disable ASPM; OS doesn't have ASPM control [ 9.184454] mpt2sas_cm0: sending port enable !! [ 9.184729] mpt2sas_cm1: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (32511224 kB) [ 9.187504] mpt2sas_cm0: host_add: handle(0x0001), sas_addr(0x500605b005bb2be0), phys(8) [ 9.190157] mpt2sas_cm0: expander_add: handle(0x0009), parent(0x0001), sas_addr(0x500304800050553f), phys(38) [ 9.191485] igb 0000:02:00.3: irq 91 for MSI/MSI-X [ 9.191552] igb 0000:02:00.3: irq 91 for MSI/MSI-X [ 9.191591] igb 0000:02:00.3: irq 92 for MSI/MSI-X [ 9.191614] igb 0000:02:00.3: irq 93 for MSI/MSI-X [ 9.191648] igb 0000:02:00.3: irq 94 for MSI/MSI-X [ 9.191680] igb 0000:02:00.3: irq 95 for MSI/MSI-X [ 9.191718] igb 0000:02:00.3: irq 96 for MSI/MSI-X [ 9.191741] igb 0000:02:00.3: irq 97 for MSI/MSI-X [ 9.191776] igb 0000:02:00.3: irq 98 for MSI/MSI-X [ 9.191803] igb 0000:02:00.3: irq 99 for MSI/MSI-X [ 9.208734] mpt2sas_cm0: expander_add: handle(0x0020), parent(0x0002), sas_addr(0x500304800040543f), phys(38) [ 9.228341] mpt2sas_cm0: port enable: SUCCESS [ 9.229445] scsi 0:0:0:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.229451] scsi 0:0:0:0: SSP: handle(0x000a), sas_addr(0x5000c5008429daa5), phy(8), device_name(0x5000c5008429daa4) [ 9.229453] scsi 0:0:0:0: SSP: enclosure_logical_id(0x500304800050553f), slot(0) [ 9.229456] scsi 0:0:0:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.238297] mpt2sas_cm1: MSI-X vectors supported: 16, no of cores: 32, max_msix_vectors: -1 [ 9.238374] mpt2sas 0000:82:00.0: irq 100 for MSI/MSI-X [ 9.238409] mpt2sas 0000:82:00.0: irq 101 for MSI/MSI-X [ 9.238436] mpt2sas 0000:82:00.0: irq 102 for MSI/MSI-X [ 9.238462] mpt2sas 0000:82:00.0: irq 103 for MSI/MSI-X [ 9.238489] mpt2sas 0000:82:00.0: irq 104 for MSI/MSI-X [ 9.238515] mpt2sas 0000:82:00.0: irq 105 for MSI/MSI-X [ 9.238547] mpt2sas 0000:82:00.0: irq 106 for MSI/MSI-X [ 9.238574] mpt2sas 0000:82:00.0: irq 107 for MSI/MSI-X [ 9.238620] mpt2sas 0000:82:00.0: irq 108 for MSI/MSI-X [ 9.238646] mpt2sas 0000:82:00.0: irq 109 for MSI/MSI-X [ 9.238672] mpt2sas 0000:82:00.0: irq 110 for MSI/MSI-X [ 9.238699] mpt2sas 0000:82:00.0: irq 111 for MSI/MSI-X [ 9.238725] mpt2sas 0000:82:00.0: irq 112 for MSI/MSI-X [ 9.238751] mpt2sas 0000:82:00.0: irq 113 for MSI/MSI-X [ 9.238781] mpt2sas 0000:82:00.0: irq 114 for MSI/MSI-X [ 9.238808] mpt2sas 0000:82:00.0: irq 115 for MSI/MSI-X [ 9.238969] mpt2sas1-msix0: PCI-MSI-X enabled: IRQ 100 [ 9.238970] mpt2sas1-msix1: PCI-MSI-X enabled: IRQ 101 [ 9.238970] mpt2sas1-msix2: PCI-MSI-X enabled: IRQ 102 [ 9.238971] mpt2sas1-msix3: PCI-MSI-X enabled: IRQ 103 [ 9.238972] mpt2sas1-msix4: PCI-MSI-X enabled: IRQ 104 [ 9.238972] mpt2sas1-msix5: PCI-MSI-X enabled: IRQ 105 [ 9.238973] mpt2sas1-msix6: PCI-MSI-X enabled: IRQ 106 [ 9.238974] mpt2sas1-msix7: PCI-MSI-X enabled: IRQ 107 [ 9.238974] mpt2sas1-msix8: PCI-MSI-X enabled: IRQ 108 [ 9.238975] mpt2sas1-msix9: PCI-MSI-X enabled: IRQ 109 [ 9.238976] mpt2sas1-msix10: PCI-MSI-X enabled: IRQ 110 [ 9.238977] mpt2sas1-msix11: PCI-MSI-X enabled: IRQ 111 [ 9.238977] mpt2sas1-msix12: PCI-MSI-X enabled: IRQ 112 [ 9.238978] mpt2sas1-msix13: PCI-MSI-X enabled: IRQ 113 [ 9.238979] mpt2sas1-msix14: PCI-MSI-X enabled: IRQ 114 [ 9.238979] mpt2sas1-msix15: PCI-MSI-X enabled: IRQ 115 [ 9.238981] mpt2sas_cm1: iomem(0x00000000ec140000), mapped(0xffffc90003b00000), size(65536) [ 9.238982] mpt2sas_cm1: ioport(0x000000000000c000), size(256) [ 9.245045] igb 0000:02:00.3: added PHC on eth3 [ 9.245047] igb 0000:02:00.3: Intel(R) Gigabit Ethernet Network Connection [ 9.245049] igb 0000:02:00.3: eth3: (PCIe:5.0Gb/s:Width x4) 00:1e:67:6b:23:2c [ 9.245136] igb 0000:02:00.3: eth3: PBA No: 100000-000 [ 9.245138] igb 0000:02:00.3: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) [ 9.293286] mpt2sas_cm1: sending message unit reset !! [ 9.294799] mpt2sas_cm1: message unit reset: SUCCESS [ 9.334100] [TTM] Zone kernel: Available graphics memory: 16255612 kiB [ 9.334101] [TTM] Zone dma32: Available graphics memory: 2097152 kiB [ 9.334102] [TTM] Initializing pool allocator [ 9.334105] [TTM] Initializing DMA pool allocator [ 9.347371] ata4: SATA link down (SStatus 0 SControl 300) [ 9.347418] ata1: SATA link down (SStatus 0 SControl 300) [ 9.347447] ata3: SATA link down (SStatus 0 SControl 300) [ 9.347480] ata2: SATA link down (SStatus 0 SControl 300) [ 9.349371] ata5: SATA link down (SStatus 0 SControl 300) [ 9.349418] ata6: SATA link down (SStatus 0 SControl 300) [ 9.356335] mpt2sas_cm1: Allocated physical memory: size(18690 kB) [ 9.356336] mpt2sas_cm1: Current Controller Queue Depth(10104),Max Controller Queue Depth(10240) [ 9.356336] mpt2sas_cm1: Scatter Gather Elements per IO(128) [ 9.370691] fbcon: mgadrmfb (fb0) is primary device [ 9.394894] scsi 0:0:1:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.394901] scsi 0:0:1:0: SSP: handle(0x000b), sas_addr(0x5000c5008429d8b1), phy(9), device_name(0x5000c5008429d8b0) [ 9.394904] scsi 0:0:1:0: SSP: enclosure_logical_id(0x500304800050553f), slot(1) [ 9.394908] scsi 0:0:1:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.401066] mpt2sas_cm1: LSISAS2308: FWVersion(20.00.07.00), ChipRevision(0x05), BiosVersion(07.39.00.00) [ 9.401068] mpt2sas_cm1: Protocol=( [ 9.401069] Initiator [ 9.401070] ,Target [ 9.401071] ), [ 9.401071] Capabilities=( [ 9.401072] TLR [ 9.401073] ,EEDP [ 9.401074] ,Snapshot Buffer [ 9.401075] ,Diag Trace Buffer [ 9.401075] ,Task Set Full [ 9.401076] ,NCQ [ 9.401076] ) [ 9.401129] scsi host8: Fusion MPT SAS Host [ 9.401572] mpt2sas_cm1: sending port enable !! [ 9.404922] mpt2sas_cm1: host_add: handle(0x0001), sas_addr(0x500605b00a6d91d0), phys(8) [ 9.407631] mpt2sas_cm1: expander_add: handle(0x0009), parent(0x0001), sas_addr(0x50030480005270bf), phys(38) [ 9.412918] scsi 0:0:2:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.412926] scsi 0:0:2:0: SSP: handle(0x000c), sas_addr(0x5000c5008429f0a5), phy(10), device_name(0x5000c5008429f0a4) [ 9.412928] scsi 0:0:2:0: SSP: enclosure_logical_id(0x500304800050553f), slot(2) [ 9.412932] scsi 0:0:2:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.427710] mpt2sas_cm1: expander_add: handle(0x0020), parent(0x0002), sas_addr(0x50030480004053bf), phys(38) [ 9.430998] scsi 0:0:3:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.431005] scsi 0:0:3:0: SSP: handle(0x000d), sas_addr(0x5000c5008429ea75), phy(11), device_name(0x5000c5008429ea74) [ 9.431015] scsi 0:0:3:0: SSP: enclosure_logical_id(0x500304800050553f), slot(3) [ 9.431019] scsi 0:0:3:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.448942] scsi 0:0:4:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.448968] scsi 0:0:4:0: SSP: handle(0x000e), sas_addr(0x5000c500842a9fad), phy(12), device_name(0x5000c500842a9fac) [ 9.448971] scsi 0:0:4:0: SSP: enclosure_logical_id(0x500304800050553f), slot(4) [ 9.448975] scsi 0:0:4:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.456344] mpt2sas_cm1: port enable: SUCCESS [ 9.457561] scsi 8:0:0:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.457568] scsi 8:0:0:0: SSP: handle(0x000a), sas_addr(0x5000c500842a08ba), phy(8), device_name(0x5000c500842a08b8) [ 9.457570] scsi 8:0:0:0: SSP: enclosure_logical_id(0x500304800052703f), slot(0) [ 9.457574] scsi 8:0:0:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.487958] scsi 8:0:1:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.487965] scsi 8:0:1:0: SSP: handle(0x000b), sas_addr(0x5000c500842a6922), phy(9), device_name(0x5000c500842a6920) [ 9.487967] scsi 8:0:1:0: SSP: enclosure_logical_id(0x500304800052703f), slot(1) [ 9.487971] scsi 8:0:1:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.488131] scsi 0:0:5:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.488137] scsi 0:0:5:0: SSP: handle(0x000f), sas_addr(0x5000c500842a058d), phy(13), device_name(0x5000c500842a058c) [ 9.488139] scsi 0:0:5:0: SSP: enclosure_logical_id(0x500304800050553f), slot(5) [ 9.488142] scsi 0:0:5:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.502946] scsi 0:0:6:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.502951] scsi 0:0:6:0: SSP: handle(0x0010), sas_addr(0x5000c500842a4405), phy(14), device_name(0x5000c500842a4404) [ 9.502953] scsi 0:0:6:0: SSP: enclosure_logical_id(0x500304800050553f), slot(6) [ 9.502957] scsi 0:0:6:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.503158] scsi 8:0:2:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.503166] scsi 8:0:2:0: SSP: handle(0x000c), sas_addr(0x5000c500842a6ad6), phy(10), device_name(0x5000c500842a6ad4) [ 9.503168] scsi 8:0:2:0: SSP: enclosure_logical_id(0x500304800052703f), slot(2) [ 9.503172] scsi 8:0:2:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.518201] scsi 0:0:7:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.518208] scsi 0:0:7:0: SSP: handle(0x0011), sas_addr(0x5000c500842a52dd), phy(15), device_name(0x5000c500842a52dc) [ 9.518210] scsi 0:0:7:0: SSP: enclosure_logical_id(0x500304800050553f), slot(7) [ 9.518214] scsi 0:0:7:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.518267] scsi 8:0:3:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.518275] scsi 8:0:3:0: SSP: handle(0x000d), sas_addr(0x5000c5008429f8ee), phy(11), device_name(0x5000c5008429f8ec) [ 9.518277] scsi 8:0:3:0: SSP: enclosure_logical_id(0x500304800052703f), slot(3) [ 9.518288] scsi 8:0:3:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.535997] scsi 8:0:4:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.536005] scsi 8:0:4:0: SSP: handle(0x000e), sas_addr(0x5000c500842a486e), phy(12), device_name(0x5000c500842a486c) [ 9.536008] scsi 8:0:4:0: SSP: enclosure_logical_id(0x500304800052703f), slot(4) [ 9.536012] scsi 8:0:4:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.536265] scsi 0:0:8:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.536272] scsi 0:0:8:0: SSP: handle(0x0012), sas_addr(0x5000c500842a9f59), phy(16), device_name(0x5000c500842a9f58) [ 9.536275] scsi 0:0:8:0: SSP: enclosure_logical_id(0x500304800050553f), slot(8) [ 9.536288] scsi 0:0:8:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.551111] scsi 0:0:9:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.551117] scsi 0:0:9:0: SSP: handle(0x0013), sas_addr(0x5000c500842a6bed), phy(17), device_name(0x5000c500842a6bec) [ 9.551119] scsi 0:0:9:0: SSP: enclosure_logical_id(0x500304800050553f), slot(9) [ 9.551122] scsi 0:0:9:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.551211] scsi 8:0:5:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.551218] scsi 8:0:5:0: SSP: handle(0x000f), sas_addr(0x5000c500842a692e), phy(13), device_name(0x5000c500842a692c) [ 9.551221] scsi 8:0:5:0: SSP: enclosure_logical_id(0x500304800052703f), slot(5) [ 9.551225] scsi 8:0:5:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.575546] scsi 0:0:10:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.575553] scsi 0:0:10:0: SSP: handle(0x0014), sas_addr(0x5000c500842a4041), phy(18), device_name(0x5000c500842a4040) [ 9.575555] scsi 0:0:10:0: SSP: enclosure_logical_id(0x500304800050553f), slot(10) [ 9.575559] scsi 0:0:10:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.575586] scsi 8:0:6:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.575594] scsi 8:0:6:0: SSP: handle(0x0010), sas_addr(0x5000c5008429f81e), phy(14), device_name(0x5000c5008429f81c) [ 9.575597] scsi 8:0:6:0: SSP: enclosure_logical_id(0x500304800052703f), slot(6) [ 9.575601] scsi 8:0:6:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.603432] Console: switching to colour frame buffer device 128x48 [ 9.654503] mgag200 0000:09:00.0: fb0: mgadrmfb frame buffer device [ 9.666350] [drm] Initialized mgag200 1.0.0 20110418 for 0000:09:00.0 on minor 0 [ 9.694959] scsi 8:0:7:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.694966] scsi 8:0:7:0: SSP: handle(0x0011), sas_addr(0x5000c500842a097a), phy(15), device_name(0x5000c500842a0978) [ 9.694982] scsi 8:0:7:0: SSP: enclosure_logical_id(0x500304800052703f), slot(7) [ 9.694984] scsi 8:0:7:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 9.694987] scsi 0:0:11:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 9.694992] scsi 0:0:11:0: SSP: handle(0x0015), sas_addr(0x5000c5008429e625), phy(19), device_name(0x5000c5008429e624) [ 9.694994] scsi 0:0:11:0: SSP: enclosure_logical_id(0x500304800050553f), slot(11) [ 9.694997] scsi 0:0:11:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 10.537380] sas: phy-1:0 added to port-1:0, phy_mask:0x1 (5001e676b2329000) [ 10.537403] sas: phy-1:1 added to port-1:1, phy_mask:0x2 (5001e676b2329001) [ 10.537418] sas: DOING DISCOVERY on port 0, pid:6 [ 10.537449] sas: DONE DISCOVERY on port 0, pid:6, result:0 [ 10.537462] sas: DOING DISCOVERY on port 1, pid:6 [ 10.537477] sas: DONE DISCOVERY on port 1, pid:6, result:0 [ 10.537486] sas: Enter sas_scsi_recover_host busy: 0 failed: 0 [ 10.537628] sas: ata7: end_device-1:0: dev error handler [ 10.698016] ata7.00: ATA-8: ST500NM0011, SN02, max UDMA/133 [ 10.698018] ata7.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32) [ 10.698726] ata7.00: configured for UDMA/133 [ 10.698744] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1 [ 10.709628] scsi 1:0:0:0: Direct-Access ATA ST500NM0011 SN02 PQ: 0 ANSI: 5 [ 11.168356] scsi 0:0:12:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 11.168663] scsi 8:0:8:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 11.168671] scsi 8:0:8:0: SSP: handle(0x0012), sas_addr(0x5000c500842a6e6e), phy(16), device_name(0x5000c500842a6e6c) [ 11.168673] scsi 8:0:8:0: SSP: enclosure_logical_id(0x500304800052703f), slot(8) [ 11.168677] scsi 8:0:8:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 11.221883] scsi 0:0:12:0: SSP: handle(0x0016), sas_addr(0x5000c500842a1481), phy(20), device_name(0x5000c500842a1480) [ 11.221884] scsi 0:0:12:0: SSP: enclosure_logical_id(0x500304800050553f), slot(12) [ 11.221888] scsi 0:0:12:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 15.480555] mlx4_core 0000:04:00.0: Old device ETS support detected [ 15.488579] mlx4_core 0000:04:00.0: Consider upgrading device FW. [ 15.496383] mlx4_core 0000:04:00.0: DMFS high rate mode not supported [ 15.504566] mlx4_core: device is working in RoCE mode: Roce V1 [ 15.512063] mlx4_core: UD QP Gid type is: V1 [ 16.094213] sas: Enter sas_scsi_recover_host busy: 0 failed: 0 [ 16.094222] sas: ata7: end_device-1:0: dev error handler [ 16.094238] sas: ata8: end_device-1:1: dev error handler [ 16.095050] scsi 8:0:9:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 16.106804] scsi 8:0:9:0: SSP: handle(0x0013), sas_addr(0x5000c500842a24c6), phy(17), device_name(0x5000c500842a24c4) [ 16.121021] scsi 8:0:9:0: SSP: enclosure_logical_id(0x500304800052703f), slot(9) [ 16.131714] scsi 8:0:9:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 16.254947] ata8.00: ATA-8: ST500NM0011, SN02, max UDMA/133 [ 16.262123] ata8.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32) [ 16.271315] ata8.00: configured for UDMA/133 [ 16.277031] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0 tries: 1 [ 16.287733] scsi 1:0:1:0: Direct-Access ATA ST500NM0011 SN02 PQ: 0 ANSI: 5 [ 16.550224] scsi 0:0:13:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 16.560293] scsi 0:0:13:0: SSP: handle(0x0017), sas_addr(0x5000c5008429ecf9), phy(21), device_name(0x5000c5008429ecf8) [ 16.573170] scsi 0:0:13:0: SSP: enclosure_logical_id(0x500304800050553f), slot(13) [ 16.576968] mlx4_core 0000:04:00.0: PCIe link speed is 8.0GT/s, device supports 8.0GT/s [ 16.576970] mlx4_core 0000:04:00.0: PCIe link width is x8, device supports x8 [ 16.577139] mlx4_core 0000:04:00.0: irq 116 for MSI/MSI-X [ 16.577160] mlx4_core 0000:04:00.0: irq 117 for MSI/MSI-X [ 16.577180] mlx4_core 0000:04:00.0: irq 118 for MSI/MSI-X [ 16.577202] mlx4_core 0000:04:00.0: irq 119 for MSI/MSI-X [ 16.577243] mlx4_core 0000:04:00.0: irq 120 for MSI/MSI-X [ 16.577266] mlx4_core 0000:04:00.0: irq 121 for MSI/MSI-X [ 16.577285] mlx4_core 0000:04:00.0: irq 122 for MSI/MSI-X [ 16.577304] mlx4_core 0000:04:00.0: irq 123 for MSI/MSI-X [ 16.577324] mlx4_core 0000:04:00.0: irq 124 for MSI/MSI-X [ 16.577343] mlx4_core 0000:04:00.0: irq 125 for MSI/MSI-X [ 16.577362] mlx4_core 0000:04:00.0: irq 126 for MSI/MSI-X [ 16.577381] mlx4_core 0000:04:00.0: irq 127 for MSI/MSI-X [ 16.577406] mlx4_core 0000:04:00.0: irq 128 for MSI/MSI-X [ 16.577429] mlx4_core 0000:04:00.0: irq 129 for MSI/MSI-X [ 16.577448] mlx4_core 0000:04:00.0: irq 130 for MSI/MSI-X [ 16.577467] mlx4_core 0000:04:00.0: irq 131 for MSI/MSI-X [ 16.577486] mlx4_core 0000:04:00.0: irq 132 for MSI/MSI-X [ 16.577505] mlx4_core 0000:04:00.0: irq 133 for MSI/MSI-X [ 16.577525] mlx4_core 0000:04:00.0: irq 134 for MSI/MSI-X [ 16.577546] mlx4_core 0000:04:00.0: irq 135 for MSI/MSI-X [ 16.577587] mlx4_core 0000:04:00.0: irq 136 for MSI/MSI-X [ 16.577609] mlx4_core 0000:04:00.0: irq 137 for MSI/MSI-X [ 16.577628] mlx4_core 0000:04:00.0: irq 138 for MSI/MSI-X [ 16.577647] mlx4_core 0000:04:00.0: irq 139 for MSI/MSI-X [ 16.577667] mlx4_core 0000:04:00.0: irq 140 for MSI/MSI-X [ 16.577685] mlx4_core 0000:04:00.0: irq 141 for MSI/MSI-X [ 16.577705] mlx4_core 0000:04:00.0: irq 142 for MSI/MSI-X [ 16.577724] mlx4_core 0000:04:00.0: irq 143 for MSI/MSI-X [ 16.577743] mlx4_core 0000:04:00.0: irq 144 for MSI/MSI-X [ 16.577765] mlx4_core 0000:04:00.0: irq 145 for MSI/MSI-X [ 16.577784] mlx4_core 0000:04:00.0: irq 146 for MSI/MSI-X [ 16.577803] mlx4_core 0000:04:00.0: irq 147 for MSI/MSI-X [ 16.577822] mlx4_core 0000:04:00.0: irq 148 for MSI/MSI-X [ 16.577841] mlx4_core 0000:04:00.0: irq 149 for MSI/MSI-X [ 16.577860] mlx4_core 0000:04:00.0: irq 150 for MSI/MSI-X [ 16.577879] mlx4_core 0000:04:00.0: irq 151 for MSI/MSI-X [ 16.577919] mlx4_core 0000:04:00.0: irq 152 for MSI/MSI-X [ 16.577941] mlx4_core 0000:04:00.0: irq 153 for MSI/MSI-X [ 16.577960] mlx4_core 0000:04:00.0: irq 154 for MSI/MSI-X [ 16.577979] mlx4_core 0000:04:00.0: irq 155 for MSI/MSI-X [ 16.577999] mlx4_core 0000:04:00.0: irq 156 for MSI/MSI-X [ 16.578018] mlx4_core 0000:04:00.0: irq 157 for MSI/MSI-X [ 16.578037] mlx4_core 0000:04:00.0: irq 158 for MSI/MSI-X [ 16.578056] mlx4_core 0000:04:00.0: irq 159 for MSI/MSI-X [ 16.578075] mlx4_core 0000:04:00.0: irq 160 for MSI/MSI-X [ 16.578097] mlx4_core 0000:04:00.0: irq 161 for MSI/MSI-X [ 16.578116] mlx4_core 0000:04:00.0: irq 162 for MSI/MSI-X [ 16.578135] mlx4_core 0000:04:00.0: irq 163 for MSI/MSI-X [ 16.578154] mlx4_core 0000:04:00.0: irq 164 for MSI/MSI-X [ 16.578173] mlx4_core 0000:04:00.0: irq 165 for MSI/MSI-X [ 16.578192] mlx4_core 0000:04:00.0: irq 166 for MSI/MSI-X [ 16.578213] mlx4_core 0000:04:00.0: irq 167 for MSI/MSI-X [ 16.578259] mlx4_core 0000:04:00.0: irq 168 for MSI/MSI-X [ 16.578283] mlx4_core 0000:04:00.0: irq 169 for MSI/MSI-X [ 16.578302] mlx4_core 0000:04:00.0: irq 170 for MSI/MSI-X [ 16.578323] mlx4_core 0000:04:00.0: irq 171 for MSI/MSI-X [ 16.578342] mlx4_core 0000:04:00.0: irq 172 for MSI/MSI-X [ 16.578361] mlx4_core 0000:04:00.0: irq 173 for MSI/MSI-X [ 16.578380] mlx4_core 0000:04:00.0: irq 174 for MSI/MSI-X [ 16.578399] mlx4_core 0000:04:00.0: irq 175 for MSI/MSI-X [ 16.578418] mlx4_core 0000:04:00.0: irq 176 for MSI/MSI-X [ 16.578443] mlx4_core 0000:04:00.0: irq 177 for MSI/MSI-X [ 16.578462] mlx4_core 0000:04:00.0: irq 178 for MSI/MSI-X [ 16.578481] mlx4_core 0000:04:00.0: irq 179 for MSI/MSI-X [ 16.578500] mlx4_core 0000:04:00.0: irq 180 for MSI/MSI-X [ 16.601435] scsi 0:0:13:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 16.817223] mlx4_ib_add: mlx4_ib: Mellanox ConnectX InfiniBand driver v4.2-1.0.0 [ 16.822508] scsi 0:0:14:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 16.822512] scsi 0:0:14:0: SSP: handle(0x0018), sas_addr(0x5000c500842a5ea5), phy(22), device_name(0x5000c500842a5ea4) [ 16.822513] scsi 0:0:14:0: SSP: enclosure_logical_id(0x500304800050553f), slot(14) [ 16.822515] scsi 0:0:14:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 16.822753] scsi 8:0:10:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 16.822757] scsi 8:0:10:0: SSP: handle(0x0014), sas_addr(0x5000c500842a5d7e), phy(18), device_name(0x5000c500842a5d7c) [ 16.822759] scsi 8:0:10:0: SSP: enclosure_logical_id(0x500304800052703f), slot(10) [ 16.822761] scsi 8:0:10:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 16.918869] mlx4_ib_add: counter index 0 for port 1 allocated 0 [ 16.927523] mlx4_ib_add: counter index 1 for port 2 allocated 0 [ 16.936916] sd 1:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB) [ 16.936943] sd 1:0:1:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB) [ 16.936970] sd 1:0:1:0: [sdb] Write Protect is off [ 16.936972] sd 1:0:1:0: [sdb] Mode Sense: 00 3a 00 00 [ 16.936983] sd 1:0:1:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 16.944397] sdb: [ 16.944578] sd 1:0:1:0: [sdb] Attached SCSI disk [ 16.982929] sd 1:0:0:0: [sda] Write Protect is off [ 16.989308] sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00 [ 16.989327] sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 17.015476] sda: sda1 sda2 [ 17.016036] sd 1:0:0:0: [sda] Attached SCSI disk [ 17.036777] scsi 8:0:11:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.037741] scsi 0:0:15:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.037746] scsi 0:0:15:0: SSP: handle(0x0019), sas_addr(0x5000c5008429fa19), phy(23), device_name(0x5000c5008429fa18) [ 17.037748] scsi 0:0:15:0: SSP: enclosure_logical_id(0x500304800050553f), slot(15) [ 17.037751] scsi 0:0:15:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.095134] scsi 8:0:11:0: SSP: handle(0x0015), sas_addr(0x5000c5008429e7ee), phy(19), device_name(0x5000c5008429e7ec) [ 17.108672] scsi 8:0:11:0: SSP: enclosure_logical_id(0x500304800052703f), slot(11) [ 17.118727] scsi 8:0:11:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.143036] scsi 8:0:12:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.143057] scsi 0:0:16:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.143066] scsi 0:0:16:0: SSP: handle(0x001a), sas_addr(0x5000c500842a6019), phy(24), device_name(0x5000c500842a6018) [ 17.143069] scsi 0:0:16:0: SSP: enclosure_logical_id(0x500304800050553f), slot(16) [ 17.143073] scsi 0:0:16:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.200742] scsi 8:0:12:0: SSP: handle(0x0016), sas_addr(0x5000c5008427aa7a), phy(20), device_name(0x5000c5008427aa78) [ 17.214170] scsi 8:0:12:0: SSP: enclosure_logical_id(0x500304800052703f), slot(12) [ 17.224142] scsi 8:0:12:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.247825] scsi 0:0:17:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.250764] scsi 8:0:13:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.250770] scsi 8:0:13:0: SSP: handle(0x0017), sas_addr(0x5000c5008429fb56), phy(21), device_name(0x5000c5008429fb54) [ 17.250772] scsi 8:0:13:0: SSP: enclosure_logical_id(0x500304800052703f), slot(13) [ 17.250776] scsi 8:0:13:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.306313] scsi 0:0:17:0: SSP: handle(0x001b), sas_addr(0x5000c5008429fb81), phy(25), device_name(0x5000c5008429fb80) [ 17.319865] scsi 0:0:17:0: SSP: enclosure_logical_id(0x500304800050553f), slot(17) [ 17.329919] scsi 0:0:17:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.356897] scsi 0:0:18:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.359912] scsi 8:0:14:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.359919] scsi 8:0:14:0: SSP: handle(0x0018), sas_addr(0x5000c5008429f216), phy(22), device_name(0x5000c5008429f214) [ 17.359922] scsi 8:0:14:0: SSP: enclosure_logical_id(0x500304800052703f), slot(14) [ 17.359925] scsi 8:0:14:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.416276] scsi 0:0:18:0: SSP: handle(0x001c), sas_addr(0x5000c500842a5d2d), phy(26), device_name(0x5000c500842a5d2c) [ 17.416278] scsi 0:0:18:0: SSP: enclosure_logical_id(0x500304800050553f), slot(18) [ 17.416283] scsi 0:0:18:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.482851] scsi 0:0:19:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.483183] scsi 8:0:15:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.483200] scsi 8:0:15:0: SSP: handle(0x0019), sas_addr(0x5000c500842a681a), phy(23), device_name(0x5000c500842a6818) [ 17.483202] scsi 8:0:15:0: SSP: enclosure_logical_id(0x500304800052703f), slot(15) [ 17.483207] scsi 8:0:15:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.538580] scsi 0:0:19:0: SSP: handle(0x001d), sas_addr(0x5000c500842a5f51), phy(27), device_name(0x5000c500842a5f50) [ 17.551735] scsi 0:0:19:0: SSP: enclosure_logical_id(0x500304800050553f), slot(19) [ 17.561436] scsi 0:0:19:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.584877] scsi 8:0:16:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.587775] scsi 0:0:20:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.587794] scsi 0:0:20:0: SSP: handle(0x001e), sas_addr(0x5000c500842a03f5), phy(28), device_name(0x5000c500842a03f4) [ 17.587796] scsi 0:0:20:0: SSP: enclosure_logical_id(0x500304800050553f), slot(20) [ 17.587799] scsi 0:0:20:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.644405] scsi 8:0:16:0: SSP: handle(0x001a), sas_addr(0x5000c500842a446e), phy(24), device_name(0x5000c500842a446c) [ 17.658110] scsi 8:0:16:0: SSP: enclosure_logical_id(0x500304800052703f), slot(16) [ 17.668330] scsi 8:0:16:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.695870] scsi 8:0:17:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.698997] scsi 0:0:21:0: Enclosure LSI SAS2X36 0e0b PQ: 0 ANSI: 5 [ 17.699003] scsi 0:0:21:0: set ignore_delay_remove for handle(0x001f) [ 17.699006] scsi 0:0:21:0: SES: handle(0x001f), sas_addr(0x500304800050553d), phy(36), device_name(0x0000000000000000) [ 17.699009] scsi 0:0:21:0: SES: enclosure_logical_id(0x500304800050553f), slot(21) [ 17.699012] scsi 0:0:21:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(6), cmd_que(1) [ 17.763985] scsi 8:0:17:0: SSP: handle(0x001b), sas_addr(0x5000c500842a6afa), phy(25), device_name(0x5000c500842a6af8) [ 17.777633] scsi 8:0:17:0: SSP: enclosure_logical_id(0x500304800052703f), slot(17) [ 17.787821] scsi 8:0:17:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.831686] scsi 0:0:22:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.831984] scsi 8:0:18:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.831991] scsi 8:0:18:0: SSP: handle(0x001c), sas_addr(0x5000c500842a5a5a), phy(26), device_name(0x5000c500842a5a58) [ 17.831994] scsi 8:0:18:0: SSP: enclosure_logical_id(0x500304800052703f), slot(18) [ 17.831998] scsi 8:0:18:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.889383] scsi 0:0:22:0: SSP: handle(0x0021), sas_addr(0x5000c500842a64cd), phy(12), device_name(0x5000c500842a64cc) [ 17.902818] scsi 0:0:22:0: SSP: enclosure_logical_id(0x500304800040543f), slot(0) [ 17.912714] scsi 0:0:22:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.937148] scsi 8:0:19:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.937185] scsi 0:0:23:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 17.937191] scsi 0:0:23:0: SSP: handle(0x0022), sas_addr(0x5000c500842a15e1), phy(13), device_name(0x5000c500842a15e0) [ 17.937193] scsi 0:0:23:0: SSP: enclosure_logical_id(0x500304800040543f), slot(1) [ 17.937196] scsi 0:0:23:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 17.996927] scsi 8:0:19:0: SSP: handle(0x001d), sas_addr(0x5000c5008429f43e), phy(27), device_name(0x5000c5008429f43c) [ 18.010687] scsi 8:0:19:0: SSP: enclosure_logical_id(0x500304800052703f), slot(19) [ 18.020958] scsi 8:0:19:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.045057] scsi 0:0:24:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.056901] scsi 0:0:24:0: SSP: handle(0x0023), sas_addr(0x5000c500842a0b71), phy(14), device_name(0x5000c500842a0b70) [ 18.058848] scsi 8:0:20:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.058856] scsi 8:0:20:0: SSP: handle(0x001e), sas_addr(0x5000c500842a189e), phy(28), device_name(0x5000c500842a189c) [ 18.058858] scsi 8:0:20:0: SSP: enclosure_logical_id(0x500304800052703f), slot(20) [ 18.058862] scsi 8:0:20:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.120480] scsi 0:0:24:0: SSP: enclosure_logical_id(0x500304800040543f), slot(2) [ 18.131235] scsi 0:0:24:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.158844] scsi 0:0:25:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.161969] scsi 8:0:21:0: Enclosure LSI SAS2X36 0e0b PQ: 0 ANSI: 5 [ 18.161987] scsi 8:0:21:0: set ignore_delay_remove for handle(0x001f) [ 18.161991] scsi 8:0:21:0: SES: handle(0x001f), sas_addr(0x50030480005270bd), phy(36), device_name(0x0000000000000000) [ 18.161994] scsi 8:0:21:0: SES: enclosure_logical_id(0x500304800052703f), slot(21) [ 18.161998] scsi 8:0:21:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(6), cmd_que(1) [ 18.227974] scsi 0:0:25:0: SSP: handle(0x0024), sas_addr(0x5000c5008429e331), phy(15), device_name(0x5000c5008429e330) [ 18.227978] scsi 0:0:25:0: SSP: enclosure_logical_id(0x500304800040543f), slot(3) [ 18.227987] scsi 0:0:25:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.297005] scsi 0:0:26:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.297030] scsi 8:0:22:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.297038] scsi 8:0:22:0: SSP: handle(0x0021), sas_addr(0x5000c500842a477e), phy(12), device_name(0x5000c500842a477c) [ 18.297041] scsi 8:0:22:0: SSP: enclosure_logical_id(0x500304800040533f), slot(0) [ 18.297045] scsi 8:0:22:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.354829] scsi 0:0:26:0: SSP: handle(0x0025), sas_addr(0x5000c500842a52fd), phy(16), device_name(0x5000c500842a52fc) [ 18.368300] scsi 0:0:26:0: SSP: enclosure_logical_id(0x500304800040543f), slot(4) [ 18.378203] scsi 0:0:26:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.403042] scsi 8:0:23:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.405819] scsi 0:0:27:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.405825] scsi 0:0:27:0: SSP: handle(0x0026), sas_addr(0x5000c5008429f051), phy(17), device_name(0x5000c5008429f050) [ 18.405827] scsi 0:0:27:0: SSP: enclosure_logical_id(0x500304800040543f), slot(5) [ 18.405830] scsi 0:0:27:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.463108] scsi 8:0:23:0: SSP: handle(0x0022), sas_addr(0x5000c500842a5cca), phy(13), device_name(0x5000c500842a5cc8) [ 18.476915] scsi 8:0:23:0: SSP: enclosure_logical_id(0x500304800040533f), slot(1) [ 18.487099] scsi 8:0:23:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.513763] scsi 8:0:24:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.516919] scsi 0:0:28:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.516926] scsi 0:0:28:0: SSP: handle(0x0027), sas_addr(0x5000c500842a19d5), phy(18), device_name(0x5000c500842a19d4) [ 18.516928] scsi 0:0:28:0: SSP: enclosure_logical_id(0x500304800040543f), slot(6) [ 18.516932] scsi 0:0:28:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.572096] scsi 8:0:24:0: SSP: handle(0x0023), sas_addr(0x5000c5008429ec5a), phy(14), device_name(0x5000c5008429ec58) [ 18.585667] scsi 8:0:24:0: SSP: enclosure_logical_id(0x500304800040533f), slot(2) [ 18.595682] scsi 8:0:24:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.619740] scsi 8:0:25:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.623911] scsi 0:0:29:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.623925] scsi 0:0:29:0: SSP: handle(0x0028), sas_addr(0x5000c500842a280d), phy(19), device_name(0x5000c500842a280c) [ 18.623927] scsi 0:0:29:0: SSP: enclosure_logical_id(0x500304800040543f), slot(7) [ 18.623931] scsi 0:0:29:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.677504] scsi 8:0:25:0: SSP: handle(0x0024), sas_addr(0x5000c500842a1342), phy(15), device_name(0x5000c500842a1340) [ 18.690972] scsi 8:0:25:0: SSP: enclosure_logical_id(0x500304800040533f), slot(3) [ 18.700886] scsi 8:0:25:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.726992] scsi 8:0:26:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.727165] scsi 0:0:30:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.727172] scsi 0:0:30:0: SSP: handle(0x0029), sas_addr(0x5000c500842a6c09), phy(20), device_name(0x5000c500842a6c08) [ 18.727175] scsi 0:0:30:0: SSP: enclosure_logical_id(0x500304800040543f), slot(8) [ 18.727187] scsi 0:0:30:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.784061] scsi 8:0:26:0: SSP: handle(0x0025), sas_addr(0x5000c500842a5b82), phy(16), device_name(0x5000c500842a5b80) [ 18.797447] scsi 8:0:26:0: SSP: enclosure_logical_id(0x500304800040533f), slot(4) [ 18.807276] scsi 8:0:26:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.831071] scsi 0:0:31:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.842909] scsi 0:0:31:0: SSP: handle(0x002a), sas_addr(0x5000c50084298f01), phy(21), device_name(0x5000c50084298f00) [ 18.844812] scsi 8:0:27:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.844819] scsi 8:0:27:0: SSP: handle(0x0026), sas_addr(0x5000c5008429e56e), phy(17), device_name(0x5000c5008429e56c) [ 18.844821] scsi 8:0:27:0: SSP: enclosure_logical_id(0x500304800040533f), slot(5) [ 18.844825] scsi 8:0:27:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.906185] scsi 0:0:31:0: SSP: enclosure_logical_id(0x500304800040543f), slot(9) [ 18.916866] scsi 0:0:31:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 18.945095] scsi 0:0:32:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.956648] scsi 0:0:32:0: SSP: handle(0x002b), sas_addr(0x5000c500842a0d41), phy(22), device_name(0x5000c500842a0d40) [ 18.959015] scsi 8:0:28:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 18.959022] scsi 8:0:28:0: SSP: handle(0x0027), sas_addr(0x5000c5008429d276), phy(18), device_name(0x5000c5008429d274) [ 18.959024] scsi 8:0:28:0: SSP: enclosure_logical_id(0x500304800040533f), slot(6) [ 18.959028] scsi 8:0:28:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.019428] scsi 0:0:32:0: SSP: enclosure_logical_id(0x500304800040543f), slot(10) [ 19.030175] scsi 0:0:32:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.076970] scsi 0:0:33:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.077293] scsi 8:0:29:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.077301] scsi 8:0:29:0: SSP: handle(0x0028), sas_addr(0x5000c500842a36f2), phy(19), device_name(0x5000c500842a36f0) [ 19.077304] scsi 8:0:29:0: SSP: enclosure_logical_id(0x500304800040533f), slot(7) [ 19.077308] scsi 8:0:29:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.134930] scsi 0:0:33:0: SSP: handle(0x002c), sas_addr(0x5000c500842a02c1), phy(23), device_name(0x5000c500842a02c0) [ 19.148437] scsi 0:0:33:0: SSP: enclosure_logical_id(0x500304800040543f), slot(11) [ 19.158485] scsi 0:0:33:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.178910] scsi 8:0:30:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.190741] scsi 8:0:30:0: SSP: handle(0x0029), sas_addr(0x5000c500842a5a2e), phy(20), device_name(0x5000c500842a5a2c) [ 19.205056] scsi 8:0:30:0: SSP: enclosure_logical_id(0x500304800040533f), slot(8) [ 19.215749] scsi 8:0:30:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.230962] scsi 0:0:34:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.230969] scsi 0:0:34:0: SSP: handle(0x002d), sas_addr(0x5000c500842a1fd5), phy(24), device_name(0x5000c500842a1fd4) [ 19.230972] scsi 0:0:34:0: SSP: enclosure_logical_id(0x500304800040543f), slot(12) [ 19.230975] scsi 0:0:34:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.307993] scsi 8:0:31:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.319865] scsi 8:0:31:0: SSP: handle(0x002a), sas_addr(0x5000c5008429f156), phy(21), device_name(0x5000c5008429f154) [ 19.321893] scsi 0:0:35:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.321901] scsi 0:0:35:0: SSP: handle(0x002e), sas_addr(0x5000c500842a1ef9), phy(25), device_name(0x5000c500842a1ef8) [ 19.321904] scsi 0:0:35:0: SSP: enclosure_logical_id(0x500304800040543f), slot(13) [ 19.321908] scsi 0:0:35:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.383166] scsi 8:0:31:0: SSP: enclosure_logical_id(0x500304800040533f), slot(9) [ 19.393821] scsi 8:0:31:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.422011] scsi 8:0:32:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.434000] scsi 8:0:32:0: SSP: handle(0x002b), sas_addr(0x5000c5008429f4c6), phy(22), device_name(0x5000c5008429f4c4) [ 19.435818] scsi 0:0:36:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.435825] scsi 0:0:36:0: SSP: handle(0x002f), sas_addr(0x5000c500842a31d9), phy(26), device_name(0x5000c500842a31d8) [ 19.435827] scsi 0:0:36:0: SSP: enclosure_logical_id(0x500304800040543f), slot(14) [ 19.435831] scsi 0:0:36:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.498426] scsi 8:0:32:0: SSP: enclosure_logical_id(0x500304800040533f), slot(10) [ 19.509500] scsi 8:0:32:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.538025] scsi 8:0:33:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.538735] scsi 0:0:37:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.538740] scsi 0:0:37:0: SSP: handle(0x0030), sas_addr(0x5000c5008429eff9), phy(27), device_name(0x5000c5008429eff8) [ 19.538742] scsi 0:0:37:0: SSP: enclosure_logical_id(0x500304800040543f), slot(15) [ 19.538745] scsi 0:0:37:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.598552] scsi 8:0:33:0: SSP: handle(0x002c), sas_addr(0x5000c500842a64a6), phy(23), device_name(0x5000c500842a64a4) [ 19.612438] scsi 8:0:33:0: SSP: enclosure_logical_id(0x500304800040533f), slot(11) [ 19.622841] scsi 8:0:33:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.646806] scsi 0:0:38:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.649797] scsi 8:0:34:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.649804] scsi 8:0:34:0: SSP: handle(0x002d), sas_addr(0x5000c5008429f2a6), phy(24), device_name(0x5000c5008429f2a4) [ 19.649806] scsi 8:0:34:0: SSP: enclosure_logical_id(0x500304800040533f), slot(12) [ 19.649810] scsi 8:0:34:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.707211] scsi 0:0:38:0: SSP: handle(0x0031), sas_addr(0x5000c500842a5025), phy(28), device_name(0x5000c500842a5024) [ 19.721004] scsi 0:0:38:0: SSP: enclosure_logical_id(0x500304800040543f), slot(16) [ 19.731325] scsi 0:0:38:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.756425] scsi 0:0:39:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.767892] scsi 0:0:39:0: SSP: handle(0x0032), sas_addr(0x5000c500842a346d), phy(29), device_name(0x5000c500842a346c) [ 19.782016] scsi 0:0:39:0: SSP: enclosure_logical_id(0x500304800040543f), slot(17) [ 19.792615] scsi 0:0:39:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.808095] scsi 8:0:35:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.820129] scsi 8:0:35:0: SSP: handle(0x002e), sas_addr(0x5000c500842a09d2), phy(25), device_name(0x5000c500842a09d0) [ 19.834667] scsi 8:0:35:0: SSP: enclosure_logical_id(0x500304800040533f), slot(13) [ 19.845693] scsi 8:0:35:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.867050] scsi 0:0:40:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.879003] scsi 0:0:40:0: SSP: handle(0x0033), sas_addr(0x5000c500842a3955), phy(30), device_name(0x5000c500842a3954) [ 19.893536] scsi 0:0:40:0: SSP: enclosure_logical_id(0x500304800040543f), slot(18) [ 19.904471] scsi 0:0:40:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.920070] scsi 8:0:36:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.932117] scsi 8:0:36:0: SSP: handle(0x002f), sas_addr(0x5000c500842a0c9e), phy(26), device_name(0x5000c500842a0c9c) [ 19.946564] scsi 8:0:36:0: SSP: enclosure_logical_id(0x500304800040533f), slot(14) [ 19.957594] scsi 8:0:36:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 19.979080] scsi 0:0:41:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 19.991045] scsi 0:0:41:0: SSP: handle(0x0034), sas_addr(0x5000c5008429f541), phy(31), device_name(0x5000c5008429f540) [ 20.005505] scsi 0:0:41:0: SSP: enclosure_logical_id(0x500304800040543f), slot(19) [ 20.016505] scsi 0:0:41:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.032077] scsi 8:0:37:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.032084] scsi 8:0:37:0: SSP: handle(0x0030), sas_addr(0x5000c500842a0c22), phy(27), device_name(0x5000c500842a0c20) [ 20.032087] scsi 8:0:37:0: SSP: enclosure_logical_id(0x500304800040533f), slot(15) [ 20.032091] scsi 8:0:37:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.113088] scsi 0:0:42:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.113145] scsi 8:0:38:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.113154] scsi 8:0:38:0: SSP: handle(0x0031), sas_addr(0x5000c5008429f7ca), phy(28), device_name(0x5000c5008429f7c8) [ 20.113157] scsi 8:0:38:0: SSP: enclosure_logical_id(0x500304800040533f), slot(16) [ 20.113171] scsi 8:0:38:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.170720] scsi 0:0:42:0: SSP: handle(0x0035), sas_addr(0x5000c5008429acbd), phy(32), device_name(0x5000c5008429acbc) [ 20.184200] scsi 0:0:42:0: SSP: enclosure_logical_id(0x500304800040543f), slot(20) [ 20.194180] scsi 0:0:42:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.218034] scsi 8:0:39:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.220802] scsi 0:0:43:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.220821] scsi 0:0:43:0: SSP: handle(0x0036), sas_addr(0x5000c5008429d531), phy(33), device_name(0x5000c5008429d530) [ 20.220823] scsi 0:0:43:0: SSP: enclosure_logical_id(0x500304800040543f), slot(21) [ 20.220826] scsi 0:0:43:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.278629] scsi 8:0:39:0: SSP: handle(0x0032), sas_addr(0x5000c500842a06e6), phy(29), device_name(0x5000c500842a06e4) [ 20.292488] scsi 8:0:39:0: SSP: enclosure_logical_id(0x500304800040533f), slot(17) [ 20.302857] scsi 8:0:39:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.329841] scsi 8:0:40:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.333001] scsi 0:0:44:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.333009] scsi 0:0:44:0: SSP: handle(0x0037), sas_addr(0x5000c500842a1c49), phy(34), device_name(0x5000c500842a1c48) [ 20.333011] scsi 0:0:44:0: SSP: enclosure_logical_id(0x500304800040543f), slot(22) [ 20.333015] scsi 0:0:44:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.388767] scsi 8:0:40:0: SSP: handle(0x0033), sas_addr(0x5000c5008429f4fe), phy(30), device_name(0x5000c5008429f4fc) [ 20.402412] scsi 8:0:40:0: SSP: enclosure_logical_id(0x500304800040533f), slot(18) [ 20.412586] scsi 8:0:40:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.454881] scsi 8:0:41:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.466881] scsi 8:0:41:0: SSP: handle(0x0034), sas_addr(0x5000c5008429fcfa), phy(31), device_name(0x5000c5008429fcf8) [ 20.468789] scsi 0:0:45:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.468796] scsi 0:0:45:0: SSP: handle(0x0038), sas_addr(0x5000c500842a4589), phy(35), device_name(0x5000c500842a4588) [ 20.468798] scsi 0:0:45:0: SSP: enclosure_logical_id(0x500304800040543f), slot(23) [ 20.468801] scsi 0:0:45:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.531003] scsi 8:0:41:0: SSP: enclosure_logical_id(0x500304800040533f), slot(19) [ 20.541962] scsi 8:0:41:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.566888] scsi 0:0:46:0: Enclosure LSI SAS2X36 0e0b PQ: 0 ANSI: 5 [ 20.569936] scsi 8:0:42:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 20.569944] scsi 8:0:42:0: SSP: handle(0x0035), sas_addr(0x5000c500842a5c9e), phy(32), device_name(0x5000c500842a5c9c) [ 20.569946] scsi 8:0:42:0: SSP: enclosure_logical_id(0x500304800040533f), slot(20) [ 20.569950] scsi 8:0:42:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 20.626447] scsi 0:0:46:0: set ignore_delay_remove for handle(0x0039) [ 20.635341] scsi 0:0:46:0: SES: handle(0x0039), sas_addr(0x500304800040543d), phy(36), device_name(0x0000000000000000) [ 20.649036] scsi 0:0:46:0: SES: enclosure_logical_id(0x500304800040543f), slot(24) [ 20.659242] scsi 0:0:46:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(6), cmd_que(1) [ 20.707298] sd 0:0:0:0: [sdc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.707608] sd 0:0:1:0: [sdd] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.707826] sd 0:0:2:0: [sde] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.708011] sd 0:0:3:0: [sdf] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.708193] sd 0:0:4:0: [sdg] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.708348] sd 0:0:5:0: [sdh] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.708560] sd 0:0:6:0: [sdi] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.708712] sd 0:0:7:0: [sdj] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.708742] sd 0:0:1:0: [sdd] Write Protect is off [ 20.708744] sd 0:0:1:0: [sdd] Mode Sense: db 00 10 08 [ 20.708938] sd 0:0:8:0: [sdk] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.708968] sd 0:0:2:0: [sde] Write Protect is off [ 20.708969] sd 0:0:2:0: [sde] Mode Sense: db 00 10 08 [ 20.709131] sd 0:0:3:0: [sdf] Write Protect is off [ 20.709133] sd 0:0:3:0: [sdf] Mode Sense: db 00 10 08 [ 20.709217] sd 0:0:9:0: [sdl] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.709274] sd 0:0:4:0: [sdg] Write Protect is off [ 20.709276] sd 0:0:4:0: [sdg] Mode Sense: db 00 10 08 [ 20.709344] sd 0:0:1:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.709457] sd 0:0:5:0: [sdh] Write Protect is off [ 20.709459] sd 0:0:5:0: [sdh] Mode Sense: db 00 10 08 [ 20.709552] sd 0:0:2:0: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.709556] sd 0:0:10:0: [sdm] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.709672] sd 0:0:6:0: [sdi] Write Protect is off [ 20.709673] sd 0:0:6:0: [sdi] Mode Sense: db 00 10 08 [ 20.709712] sd 0:0:3:0: [sdf] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.709782] sd 0:0:11:0: [sdn] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.709850] sd 0:0:7:0: [sdj] Write Protect is off [ 20.709851] sd 0:0:7:0: [sdj] Mode Sense: db 00 10 08 [ 20.709858] sd 0:0:4:0: [sdg] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.710018] sd 0:0:12:0: [sdo] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.710040] sd 0:0:5:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.710044] sd 0:0:8:0: [sdk] Write Protect is off [ 20.710045] sd 0:0:8:0: [sdk] Mode Sense: db 00 10 08 [ 20.710264] sd 0:0:13:0: [sdp] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.710294] sd 0:0:6:0: [sdi] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.710350] sd 0:0:9:0: [sdl] Write Protect is off [ 20.710352] sd 0:0:9:0: [sdl] Mode Sense: db 00 10 08 [ 20.710411] sd 0:0:14:0: [sdq] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.710437] sd 0:0:7:0: [sdj] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.710444] sd 0:0:15:0: [sdr] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.710639] sd 0:0:8:0: [sdk] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.710696] sd 0:0:10:0: [sdm] Write Protect is off [ 20.710697] sd 0:0:10:0: [sdm] Mode Sense: db 00 10 08 [ 20.710762] sd 0:0:16:0: [sds] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.710875] sd 0:0:11:0: [sdn] Write Protect is off [ 20.710876] sd 0:0:11:0: [sdn] Mode Sense: db 00 10 08 [ 20.710935] sd 0:0:9:0: [sdl] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.711113] sd 0:0:12:0: [sdo] Write Protect is off [ 20.711114] sd 0:0:12:0: [sdo] Mode Sense: db 00 10 08 [ 20.711277] sd 0:0:10:0: [sdm] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.711305] sd 0:0:17:0: [sdt] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.711385] sd 0:0:13:0: [sdp] Write Protect is off [ 20.711387] sd 0:0:13:0: [sdp] Mode Sense: db 00 10 08 [ 20.711465] sd 0:0:11:0: [sdn] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.711513] sd 0:0:18:0: [sdu] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.711514] sd 0:0:14:0: [sdq] Write Protect is off [ 20.711515] sd 0:0:14:0: [sdq] Mode Sense: db 00 10 08 [ 20.711592] sd 0:0:15:0: [sdr] Write Protect is off [ 20.711593] sd 0:0:15:0: [sdr] Mode Sense: db 00 10 08 [ 20.711700] sd 0:0:12:0: [sdo] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.711749] sd 0:0:19:0: [sdv] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.711834] sd 0:0:20:0: [sdw] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.711959] sd 0:0:16:0: [sds] Write Protect is off [ 20.711960] sd 0:0:16:0: [sds] Mode Sense: db 00 10 08 [ 20.711979] sd 0:0:13:0: [sdp] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.712101] sd 0:0:14:0: [sdq] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.712189] sd 0:0:15:0: [sdr] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.712298] sd 0:0:22:0: [sdx] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.712424] sd 0:0:17:0: [sdt] Write Protect is off [ 20.712425] sd 0:0:17:0: [sdt] Mode Sense: db 00 10 08 [ 20.712558] sd 0:0:16:0: [sds] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.712622] sd 0:0:18:0: [sdu] Write Protect is off [ 20.712624] sd 0:0:18:0: [sdu] Mode Sense: db 00 10 08 [ 20.712659] sd 0:0:23:0: [sdy] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.712832] sd 0:0:19:0: [sdv] Write Protect is off [ 20.712833] sd 0:0:19:0: [sdv] Mode Sense: db 00 10 08 [ 20.712929] sd 0:0:20:0: [sdw] Write Protect is off [ 20.712931] sd 0:0:20:0: [sdw] Mode Sense: db 00 10 08 [ 20.712942] sd 0:0:24:0: [sdz] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.713017] sd 0:0:17:0: [sdt] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.713228] sd 0:0:18:0: [sdu] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.713372] sd 0:0:22:0: [sdx] Write Protect is off [ 20.713374] sd 0:0:22:0: [sdx] Mode Sense: db 00 10 08 [ 20.713424] sd 0:0:19:0: [sdv] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.713483] sd 0:0:25:0: [sdaa] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.713544] sd 0:0:20:0: [sdw] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.713624] random: crng init done [ 20.713725] sd 0:0:23:0: [sdy] Write Protect is off [ 20.713726] sd 0:0:23:0: [sdy] Mode Sense: db 00 10 08 [ 20.713777] sd 0:0:26:0: [sdab] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.713968] sd 0:0:22:0: [sdx] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.714055] sd 0:0:27:0: [sdac] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.714070] sd 0:0:24:0: [sdz] Write Protect is off [ 20.714071] sd 0:0:24:0: [sdz] Mode Sense: db 00 10 08 [ 20.714311] sd 0:0:23:0: [sdy] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.714350] sd 0:0:28:0: [sdad] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.714389] sd 0:0:29:0: [sdae] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.714598] sd 0:0:25:0: [sdaa] Write Protect is off [ 20.714599] sd 0:0:25:0: [sdaa] Mode Sense: db 00 10 08 [ 20.714646] sd 0:0:24:0: [sdz] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.714733] sd 0:0:30:0: [sdaf] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.714873] sd 0:0:26:0: [sdab] Write Protect is off [ 20.714875] sd 0:0:26:0: [sdab] Mode Sense: db 00 10 08 [ 20.715033] sd 0:0:31:0: [sdag] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.715116] sd 0:0:27:0: [sdac] Write Protect is off [ 20.715117] sd 0:0:27:0: [sdac] Mode Sense: db 00 10 08 [ 20.715193] sd 0:0:25:0: [sdaa] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.715310] sd 0:0:32:0: [sdah] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.715313] sd 0:0:1:0: [sdd] Attached SCSI disk [ 20.715428] sd 0:0:28:0: [sdad] Write Protect is off [ 20.715430] sd 0:0:28:0: [sdad] Mode Sense: db 00 10 08 [ 20.715464] sd 0:0:26:0: [sdab] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.715491] sd 0:0:29:0: [sdae] Write Protect is off [ 20.715492] sd 0:0:29:0: [sdae] Mode Sense: db 00 10 08 [ 20.715538] sd 0:0:2:0: [sde] Attached SCSI disk [ 20.715613] sd 0:0:33:0: [sdai] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.715704] sd 0:0:27:0: [sdac] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.715731] sd 0:0:3:0: [sdf] Attached SCSI disk [ 20.715808] sd 0:0:30:0: [sdaf] Write Protect is off [ 20.715809] sd 0:0:30:0: [sdaf] Mode Sense: db 00 10 08 [ 20.715868] sd 0:0:34:0: [sdaj] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.715900] sd 0:0:4:0: [sdg] Attached SCSI disk [ 20.715937] sd 0:0:5:0: [sdh] Attached SCSI disk [ 20.716021] sd 0:0:28:0: [sdad] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.716096] sd 0:0:29:0: [sdae] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.716122] sd 0:0:31:0: [sdag] Write Protect is off [ 20.716123] sd 0:0:31:0: [sdag] Mode Sense: db 00 10 08 [ 20.716129] sd 0:0:35:0: [sdak] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.716317] sd 0:0:6:0: [sdi] Attached SCSI disk [ 20.716388] sd 0:0:32:0: [sdah] Write Protect is off [ 20.716389] sd 0:0:32:0: [sdah] Mode Sense: db 00 10 08 [ 20.716401] sd 0:0:36:0: [sdal] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.716403] sd 0:0:30:0: [sdaf] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.716429] sd 0:0:7:0: [sdj] Attached SCSI disk [ 20.716709] sd 0:0:31:0: [sdag] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.716735] sd 0:0:33:0: [sdai] Write Protect is off [ 20.716736] sd 0:0:33:0: [sdai] Mode Sense: db 00 10 08 [ 20.716758] sd 0:0:37:0: [sdam] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.716771] sd 0:0:8:0: [sdk] Attached SCSI disk [ 20.716928] sd 0:0:9:0: [sdl] Attached SCSI disk [ 20.716967] sd 0:0:32:0: [sdah] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.717015] sd 0:0:34:0: [sdaj] Write Protect is off [ 20.717016] sd 0:0:34:0: [sdaj] Mode Sense: db 00 10 08 [ 20.717030] sd 0:0:38:0: [sdan] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.717199] sd 0:0:10:0: [sdm] Attached SCSI disk [ 20.717243] sd 0:0:35:0: [sdak] Write Protect is off [ 20.717244] sd 0:0:35:0: [sdak] Mode Sense: db 00 10 08 [ 20.717316] sd 0:0:33:0: [sdai] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.717401] sd 0:0:11:0: [sdn] Attached SCSI disk [ 20.717457] sd 0:0:39:0: [sdao] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.717521] sd 0:0:36:0: [sdal] Write Protect is off [ 20.717522] sd 0:0:36:0: [sdal] Mode Sense: db 00 10 08 [ 20.717586] sd 0:0:34:0: [sdaj] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.717596] sd 0:0:12:0: [sdo] Attached SCSI disk [ 20.717654] sd 0:0:40:0: [sdap] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.717816] sd 0:0:13:0: [sdp] Attached SCSI disk [ 20.717822] sd 0:0:37:0: [sdam] Write Protect is off [ 20.717824] sd 0:0:37:0: [sdam] Mode Sense: db 00 10 08 [ 20.717841] sd 0:0:35:0: [sdak] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.717970] sd 0:0:41:0: [sdaq] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.718063] sd 0:0:14:0: [sdq] Attached SCSI disk [ 20.718142] sd 0:0:15:0: [sdr] Attached SCSI disk [ 20.718144] sd 0:0:36:0: [sdal] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.718180] sd 0:0:38:0: [sdan] Write Protect is off [ 20.718181] sd 0:0:38:0: [sdan] Mode Sense: db 00 10 08 [ 20.718234] sd 0:0:42:0: [sdar] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.718427] sd 0:0:37:0: [sdam] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.718491] sd 0:0:43:0: [sdas] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.718534] sd 0:0:16:0: [sds] Attached SCSI disk [ 20.718552] sd 0:0:39:0: [sdao] Write Protect is off [ 20.718554] sd 0:0:39:0: [sdao] Mode Sense: db 00 10 08 [ 20.718742] sd 0:0:44:0: [sdat] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.718746] sd 0:0:38:0: [sdan] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.718756] sd 0:0:40:0: [sdap] Write Protect is off [ 20.718757] sd 0:0:40:0: [sdap] Mode Sense: db 00 10 08 [ 20.718963] sd 0:0:17:0: [sdt] Attached SCSI disk [ 20.719067] sd 0:0:45:0: [sdau] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 20.719070] sd 0:0:41:0: [sdaq] Write Protect is off [ 20.719071] sd 0:0:41:0: [sdaq] Mode Sense: db 00 10 08 [ 20.719158] sd 0:0:39:0: [sdao] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.719224] sd 0:0:18:0: [sdu] Attached SCSI disk [ 20.719349] sd 0:0:40:0: [sdap] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.719358] sd 0:0:42:0: [sdar] Write Protect is off [ 20.719360] sd 0:0:42:0: [sdar] Mode Sense: db 00 10 08 [ 20.719440] sd 0:0:19:0: [sdv] Attached SCSI disk [ 20.719569] sd 0:0:43:0: [sdas] Write Protect is off [ 20.719570] sd 0:0:43:0: [sdas] Mode Sense: db 00 10 08 [ 20.719632] sd 0:0:41:0: [sdaq] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.719633] sd 0:0:20:0: [sdw] Attached SCSI disk [ 20.719830] sd 0:0:44:0: [sdat] Write Protect is off [ 20.719832] sd 0:0:44:0: [sdat] Mode Sense: db 00 10 08 [ 20.719923] sd 0:0:22:0: [sdx] Attached SCSI disk [ 20.719936] sd 0:0:42:0: [sdar] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.720124] sd 0:0:43:0: [sdas] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.720186] sd 0:0:45:0: [sdau] Write Protect is off [ 20.720187] sd 0:0:45:0: [sdau] Mode Sense: db 00 10 08 [ 20.720277] sd 0:0:23:0: [sdy] Attached SCSI disk [ 20.720389] sd 0:0:44:0: [sdat] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.720718] sd 0:0:24:0: [sdz] Attached SCSI disk [ 20.720764] sd 0:0:45:0: [sdau] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 20.721135] sd 0:0:25:0: [sdaa] Attached SCSI disk [ 20.721363] sd 0:0:26:0: [sdab] Attached SCSI disk [ 20.721716] sd 0:0:27:0: [sdac] Attached SCSI disk [ 20.722057] sd 0:0:29:0: [sdae] Attached SCSI disk [ 20.722163] sd 0:0:28:0: [sdad] Attached SCSI disk [ 20.722324] sd 0:0:30:0: [sdaf] Attached SCSI disk [ 20.722781] sd 0:0:31:0: [sdag] Attached SCSI disk [ 20.722958] sd 0:0:32:0: [sdah] Attached SCSI disk [ 20.723226] sd 0:0:33:0: [sdai] Attached SCSI disk [ 20.723583] sd 0:0:34:0: [sdaj] Attached SCSI disk [ 20.723807] sd 0:0:35:0: [sdak] Attached SCSI disk [ 20.724118] sd 0:0:36:0: [sdal] Attached SCSI disk [ 20.724299] sd 0:0:37:0: [sdam] Attached SCSI disk [ 20.724665] sd 0:0:38:0: [sdan] Attached SCSI disk [ 20.725124] sd 0:0:39:0: [sdao] Attached SCSI disk [ 20.725262] sd 0:0:40:0: [sdap] Attached SCSI disk [ 20.725424] sd 0:0:41:0: [sdaq] Attached SCSI disk [ 20.725849] sd 0:0:42:0: [sdar] Attached SCSI disk [ 20.726002] sd 0:0:43:0: [sdas] Attached SCSI disk [ 20.726212] sd 0:0:44:0: [sdat] Attached SCSI disk [ 20.726562] sd 0:0:45:0: [sdau] Attached SCSI disk [ 22.137406] sd 0:0:0:0: [sdc] Write Protect is off [ 22.139969] scsi 8:0:43:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 22.139976] scsi 8:0:43:0: SSP: handle(0x0036), sas_addr(0x5000c5008429ac96), phy(33), device_name(0x5000c5008429ac94) [ 22.139979] scsi 8:0:43:0: SSP: enclosure_logical_id(0x500304800040533f), slot(21) [ 22.139984] scsi 8:0:43:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 22.184828] sd 0:0:0:0: [sdc] Mode Sense: db 00 10 08 [ 22.185481] sd 0:0:0:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.201527] sd 0:0:0:0: [sdc] Attached SCSI disk [ 22.211085] scsi 8:0:44:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 22.221717] scsi 8:0:44:0: SSP: handle(0x0037), sas_addr(0x5000c5008429f766), phy(34), device_name(0x5000c5008429f764) [ 22.234854] scsi 8:0:44:0: SSP: enclosure_logical_id(0x500304800040533f), slot(22) [ 22.244498] scsi 8:0:44:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 22.288776] scsi 8:0:45:0: Direct-Access SEAGATE ST1000NM0023 0004 PQ: 0 ANSI: 6 [ 22.299481] scsi 8:0:45:0: SSP: handle(0x0038), sas_addr(0x5000c500842a5f16), phy(35), device_name(0x5000c500842a5f14) [ 22.312671] scsi 8:0:45:0: SSP: enclosure_logical_id(0x500304800040533f), slot(23) [ 22.322415] scsi 8:0:45:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) [ 22.348956] scsi 8:0:46:0: Enclosure LSI SAS2X36 0e0b PQ: 0 ANSI: 5 [ 22.359511] scsi 8:0:46:0: set ignore_delay_remove for handle(0x0039) [ 22.367976] scsi 8:0:46:0: SES: handle(0x0039), sas_addr(0x50030480004053bd), phy(36), device_name(0x0000000000000000) [ 22.381265] scsi 8:0:46:0: SES: enclosure_logical_id(0x500304800040533f), slot(24) [ 22.391077] scsi 8:0:46:0: qdepth(254), tagged(1), simple(0), ordered(0), scsi_level(6), cmd_que(1) [ 22.417321] sd 8:0:0:0: [sdav] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427630] sd 8:0:4:0: [sdaz] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427635] sd 8:0:6:0: [sdbb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427640] sd 8:0:1:0: [sdaw] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427642] sd 8:0:2:0: [sdax] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427647] sd 8:0:5:0: [sdba] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427655] sd 8:0:7:0: [sdbc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427669] sd 8:0:3:0: [sday] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427787] sd 8:0:8:0: [sdbd] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427825] sd 8:0:9:0: [sdbe] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427905] sd 8:0:10:0: [sdbf] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427915] sd 8:0:11:0: [sdbg] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.427984] sd 8:0:12:0: [sdbh] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428045] sd 8:0:13:0: [sdbi] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428079] sd 8:0:14:0: [sdbj] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428124] sd 8:0:15:0: [sdbk] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428150] sd 8:0:16:0: [sdbl] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428210] sd 8:0:17:0: [sdbm] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428228] sd 8:0:18:0: [sdbn] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428330] sd 8:0:19:0: [sdbo] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428335] sd 8:0:20:0: [sdbp] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428433] sd 8:0:22:0: [sdbq] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428516] sd 8:0:23:0: [sdbr] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428577] sd 8:0:24:0: [sdbs] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428622] sd 8:0:0:0: [sdav] Write Protect is off [ 22.428624] sd 8:0:0:0: [sdav] Mode Sense: db 00 10 08 [ 22.428651] sd 8:0:25:0: [sdbt] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428707] sd 8:0:26:0: [sdbu] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428790] sd 8:0:6:0: [sdbb] Write Protect is off [ 22.428793] sd 8:0:6:0: [sdbb] Mode Sense: db 00 10 08 [ 22.428837] sd 8:0:7:0: [sdbc] Write Protect is off [ 22.428839] sd 8:0:7:0: [sdbc] Mode Sense: db 00 10 08 [ 22.428859] sd 8:0:2:0: [sdax] Write Protect is off [ 22.428861] sd 8:0:2:0: [sdax] Mode Sense: db 00 10 08 [ 22.428864] sd 8:0:1:0: [sdaw] Write Protect is off [ 22.428866] sd 8:0:5:0: [sdba] Write Protect is off [ 22.428868] sd 8:0:3:0: [sday] Write Protect is off [ 22.428869] sd 8:0:1:0: [sdaw] Mode Sense: db 00 10 08 [ 22.428870] sd 8:0:5:0: [sdba] Mode Sense: db 00 10 08 [ 22.428872] sd 8:0:3:0: [sday] Mode Sense: db 00 10 08 [ 22.428873] sd 8:0:27:0: [sdbv] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428919] sd 8:0:8:0: [sdbd] Write Protect is off [ 22.428921] sd 8:0:8:0: [sdbd] Mode Sense: db 00 10 08 [ 22.428989] sd 8:0:28:0: [sdbw] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.428994] sd 8:0:9:0: [sdbe] Write Protect is off [ 22.428995] sd 8:0:9:0: [sdbe] Mode Sense: db 00 10 08 [ 22.429049] sd 8:0:10:0: [sdbf] Write Protect is off [ 22.429051] sd 8:0:10:0: [sdbf] Mode Sense: db 00 10 08 [ 22.429052] sd 8:0:29:0: [sdbx] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.429123] sd 8:0:11:0: [sdbg] Write Protect is off [ 22.429125] sd 8:0:11:0: [sdbg] Mode Sense: db 00 10 08 [ 22.429139] sd 8:0:12:0: [sdbh] Write Protect is off [ 22.429142] sd 8:0:12:0: [sdbh] Mode Sense: db 00 10 08 [ 22.429226] sd 8:0:0:0: [sdav] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429285] sd 8:0:14:0: [sdbj] Write Protect is off [ 22.429287] sd 8:0:14:0: [sdbj] Mode Sense: db 00 10 08 [ 22.429311] sd 8:0:30:0: [sdby] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.429312] sd 8:0:16:0: [sdbl] Write Protect is off [ 22.429314] sd 8:0:15:0: [sdbk] Write Protect is off [ 22.429315] sd 8:0:16:0: [sdbl] Mode Sense: db 00 10 08 [ 22.429316] sd 8:0:15:0: [sdbk] Mode Sense: db 00 10 08 [ 22.429318] sd 8:0:13:0: [sdbi] Write Protect is off [ 22.429321] sd 8:0:13:0: [sdbi] Mode Sense: db 00 10 08 [ 22.429336] sd 8:0:17:0: [sdbm] Write Protect is off [ 22.429338] sd 8:0:17:0: [sdbm] Mode Sense: db 00 10 08 [ 22.429349] sd 8:0:31:0: [sdbz] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.429376] sd 8:0:6:0: [sdbb] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429395] sd 8:0:18:0: [sdbn] Write Protect is off [ 22.429397] sd 8:0:18:0: [sdbn] Mode Sense: db 00 10 08 [ 22.429456] sd 8:0:7:0: [sdbc] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429489] sd 8:0:2:0: [sdax] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429491] sd 8:0:3:0: [sday] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429495] sd 8:0:1:0: [sdaw] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429512] sd 8:0:5:0: [sdba] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429530] sd 8:0:19:0: [sdbo] Write Protect is off [ 22.429533] sd 8:0:19:0: [sdbo] Mode Sense: db 00 10 08 [ 22.429574] sd 8:0:33:0: [sdcb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.429595] sd 8:0:9:0: [sdbe] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429607] sd 8:0:20:0: [sdbp] Write Protect is off [ 22.429610] sd 8:0:20:0: [sdbp] Mode Sense: db 00 10 08 [ 22.429654] sd 8:0:10:0: [sdbf] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429663] sd 8:0:23:0: [sdbr] Write Protect is off [ 22.429666] sd 8:0:23:0: [sdbr] Mode Sense: db 00 10 08 [ 22.429720] sd 8:0:24:0: [sdbs] Write Protect is off [ 22.429722] sd 8:0:24:0: [sdbs] Mode Sense: db 00 10 08 [ 22.429731] sd 8:0:11:0: [sdbg] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429733] sd 8:0:34:0: [sdcc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.429759] sd 8:0:12:0: [sdbh] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429816] sd 8:0:25:0: [sdbt] Write Protect is off [ 22.429818] sd 8:0:25:0: [sdbt] Mode Sense: db 00 10 08 [ 22.429893] sd 8:0:14:0: [sdbj] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429935] sd 8:0:15:0: [sdbk] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429943] sd 8:0:16:0: [sdbl] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429964] sd 8:0:13:0: [sdbi] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.429967] sd 8:0:36:0: [sdce] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.429979] sd 8:0:17:0: [sdbm] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430006] sd 8:0:22:0: [sdbq] Write Protect is off [ 22.430008] sd 8:0:22:0: [sdbq] Mode Sense: db 00 10 08 [ 22.430010] sd 8:0:26:0: [sdbu] Write Protect is off [ 22.430013] sd 8:0:26:0: [sdbu] Mode Sense: db 00 10 08 [ 22.430023] sd 8:0:35:0: [sdcd] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.430044] sd 8:0:27:0: [sdbv] Write Protect is off [ 22.430047] sd 8:0:27:0: [sdbv] Mode Sense: db 00 10 08 [ 22.430078] sd 8:0:18:0: [sdbn] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430205] sd 8:0:37:0: [sdcf] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.430249] sd 8:0:39:0: [sdch] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.430260] sd 8:0:20:0: [sdbp] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430262] sd 8:0:28:0: [sdbw] Write Protect is off [ 22.430265] sd 8:0:28:0: [sdbw] Mode Sense: db 00 10 08 [ 22.430280] sd 8:0:38:0: [sdcg] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.430357] sd 8:0:32:0: [sdca] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.430407] sd 8:0:29:0: [sdbx] Write Protect is off [ 22.430410] sd 8:0:29:0: [sdbx] Mode Sense: db 00 10 08 [ 22.430413] sd 8:0:23:0: [sdbr] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430428] sd 8:0:30:0: [sdby] Write Protect is off [ 22.430431] sd 8:0:30:0: [sdby] Mode Sense: db 00 10 08 [ 22.430480] sd 8:0:19:0: [sdbo] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430527] sd 8:0:24:0: [sdbs] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430644] sd 8:0:8:0: [sdbd] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430664] sd 8:0:31:0: [sdbz] Write Protect is off [ 22.430667] sd 8:0:31:0: [sdbz] Mode Sense: db 00 10 08 [ 22.430678] sd 8:0:27:0: [sdbv] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430711] sd 8:0:25:0: [sdbt] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430756] sd 8:0:22:0: [sdbq] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.430792] sd 8:0:33:0: [sdcb] Write Protect is off [ 22.430794] sd 8:0:33:0: [sdcb] Mode Sense: db 00 10 08 [ 22.431006] sd 8:0:30:0: [sdby] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.431015] sd 8:0:28:0: [sdbw] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.431091] sd 8:0:29:0: [sdbx] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.431201] sd 8:0:36:0: [sdce] Write Protect is off [ 22.431204] sd 8:0:36:0: [sdce] Mode Sense: db 00 10 08 [ 22.431218] sd 8:0:34:0: [sdcc] Write Protect is off [ 22.431220] sd 8:0:34:0: [sdcc] Mode Sense: db 00 10 08 [ 22.431350] sd 8:0:31:0: [sdbz] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.431400] sd 8:0:26:0: [sdbu] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.431433] sd 8:0:33:0: [sdcb] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.431448] sd 8:0:38:0: [sdcg] Write Protect is off [ 22.431450] sd 8:0:38:0: [sdcg] Mode Sense: db 00 10 08 [ 22.431608] sd 8:0:40:0: [sdci] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.431672] sd 8:0:32:0: [sdca] Write Protect is off [ 22.431674] sd 8:0:32:0: [sdca] Mode Sense: db 00 10 08 [ 22.431756] sd 8:0:41:0: [sdcj] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.431800] sd 8:0:36:0: [sdce] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.431993] sd 8:0:34:0: [sdcc] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.432151] sd 8:0:39:0: [sdch] Write Protect is off [ 22.432153] sd 8:0:39:0: [sdch] Mode Sense: db 00 10 08 [ 22.432198] sd 8:0:37:0: [sdcf] Write Protect is off [ 22.432201] sd 8:0:37:0: [sdcf] Mode Sense: db 00 10 08 [ 22.432244] sd 8:0:38:0: [sdcg] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.432286] sd 8:0:32:0: [sdca] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.432596] sd 8:0:35:0: [sdcd] Write Protect is off [ 22.432598] sd 8:0:35:0: [sdcd] Mode Sense: db 00 10 08 [ 22.432746] sd 8:0:39:0: [sdch] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.432836] sd 8:0:37:0: [sdcf] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.433094] sd 8:0:40:0: [sdci] Write Protect is off [ 22.433097] sd 8:0:40:0: [sdci] Mode Sense: db 00 10 08 [ 22.433143] sd 8:0:42:0: [sdck] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.433191] sd 8:0:41:0: [sdcj] Write Protect is off [ 22.433192] sd 8:0:41:0: [sdcj] Mode Sense: db 00 10 08 [ 22.433199] sd 8:0:35:0: [sdcd] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.433338] sd 8:0:43:0: [sdcl] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.433404] sd 8:0:44:0: [sdcm] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.433821] sd 8:0:45:0: [sdcn] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) [ 22.433853] sd 8:0:41:0: [sdcj] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.434283] sd 8:0:42:0: [sdck] Write Protect is off [ 22.434285] sd 8:0:42:0: [sdck] Mode Sense: db 00 10 08 [ 22.434516] sd 8:0:44:0: [sdcm] Write Protect is off [ 22.434518] sd 8:0:44:0: [sdcm] Mode Sense: db 00 10 08 [ 22.434521] sd 8:0:40:0: [sdci] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.434570] sd 8:0:43:0: [sdcl] Write Protect is off [ 22.434573] sd 8:0:43:0: [sdcl] Mode Sense: db 00 10 08 [ 22.434845] sd 8:0:42:0: [sdck] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.434925] sd 8:0:45:0: [sdcn] Write Protect is off [ 22.434927] sd 8:0:45:0: [sdcn] Mode Sense: db 00 10 08 [ 22.435165] sd 8:0:44:0: [sdcm] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.435185] sd 8:0:43:0: [sdcl] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.435548] sd 8:0:45:0: [sdcn] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 22.436456] sd 8:0:0:0: [sdav] Attached SCSI disk [ 22.436660] sd 8:0:2:0: [sdax] Attached SCSI disk [ 22.436745] sd 8:0:1:0: [sdaw] Attached SCSI disk [ 22.436969] sd 8:0:7:0: [sdbc] Attached SCSI disk [ 22.437312] sd 8:0:5:0: [sdba] Attached SCSI disk [ 22.437469] sd 8:0:6:0: [sdbb] Attached SCSI disk [ 22.437478] sd 8:0:9:0: [sdbe] Attached SCSI disk [ 22.437496] sd 8:0:10:0: [sdbf] Attached SCSI disk [ 22.437500] sd 8:0:3:0: [sday] Attached SCSI disk [ 22.437562] sd 8:0:15:0: [sdbk] Attached SCSI disk [ 22.437801] sd 8:0:24:0: [sdbs] Attached SCSI disk [ 22.437974] sd 8:0:11:0: [sdbg] Attached SCSI disk [ 22.438041] sd 8:0:29:0: [sdbx] Attached SCSI disk [ 22.438274] sd 8:0:17:0: [sdbm] Attached SCSI disk [ 22.438364] sd 8:0:18:0: [sdbn] Attached SCSI disk [ 22.438403] sd 8:0:13:0: [sdbi] Attached SCSI disk [ 22.438479] sd 8:0:30:0: [sdby] Attached SCSI disk [ 22.438502] sd 8:0:23:0: [sdbr] Attached SCSI disk [ 22.438563] sd 8:0:27:0: [sdbv] Attached SCSI disk [ 22.438665] sd 8:0:20:0: [sdbp] Attached SCSI disk [ 22.438668] sd 8:0:19:0: [sdbo] Attached SCSI disk [ 22.438713] sd 8:0:36:0: [sdce] Attached SCSI disk [ 22.438714] sd 8:0:12:0: [sdbh] Attached SCSI disk [ 22.438802] sd 8:0:14:0: [sdbj] Attached SCSI disk [ 22.438964] sd 8:0:31:0: [sdbz] Attached SCSI disk [ 22.439052] sd 8:0:38:0: [sdcg] Attached SCSI disk [ 22.439069] sd 8:0:34:0: [sdcc] Attached SCSI disk [ 22.439170] sd 8:0:33:0: [sdcb] Attached SCSI disk [ 22.439238] sd 8:0:16:0: [sdbl] Attached SCSI disk [ 22.439359] sd 8:0:22:0: [sdbq] Attached SCSI disk [ 22.439415] sd 8:0:8:0: [sdbd] Attached SCSI disk [ 22.439428] sd 8:0:26:0: [sdbu] Attached SCSI disk [ 22.439584] sd 8:0:28:0: [sdbw] Attached SCSI disk [ 22.439970] sd 8:0:39:0: [sdch] Attached SCSI disk [ 22.439978] sd 8:0:32:0: [sdca] Attached SCSI disk [ 22.440292] sd 8:0:37:0: [sdcf] Attached SCSI disk [ 22.440381] sd 8:0:25:0: [sdbt] Attached SCSI disk [ 22.440763] sd 8:0:35:0: [sdcd] Attached SCSI disk [ 22.441018] sd 8:0:40:0: [sdci] Attached SCSI disk [ 22.441137] sd 8:0:42:0: [sdck] Attached SCSI disk [ 22.441301] sd 8:0:44:0: [sdcm] Attached SCSI disk [ 22.441416] sd 8:0:43:0: [sdcl] Attached SCSI disk [ 22.441781] sd 8:0:41:0: [sdcj] Attached SCSI disk [ 22.441910] sd 8:0:45:0: [sdcn] Attached SCSI disk [ 23.815264] sd 8:0:4:0: [sdaz] Write Protect is off [ 23.820918] sd 8:0:4:0: [sdaz] Mode Sense: db 00 10 08 [ 23.821533] sd 8:0:4:0: [sdaz] Write cache: enabled, read cache: enabled, supports DPO and FUA [ 23.837400] sd 8:0:4:0: [sdaz] Attached SCSI disk [ 27.142381] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem [ 27.164216] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) [ 27.673197] systemd-journald[281]: Received SIGTERM from PID 1 (systemd). [ 28.241181] SELinux: Disabled at runtime. [ 28.247088] SELinux: Unregistering netfilter hooks [ 28.315118] type=1404 audit(1510797707.048:2): selinux=0 auid=4294967295 ses=4294967295 [ 28.536579] ip_tables: (C) 2000-2006 Netfilter Core Team [ 28.544217] systemd[1]: Inserted module 'ip_tables' [ 30.321028] EXT4-fs (sda1): re-mounted. Opts: (null) [ 30.336217] RPC: Registered named UNIX socket transport module. [ 30.336218] RPC: Registered udp transport module. [ 30.336219] RPC: Registered tcp transport module. [ 30.336220] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 30.513198] Installing knfsd (copyright (C) 1996 okir@monad.swb.de). [ 30.546705] systemd-journald[1013]: Received request to flush runtime journal from PID 1 [ 30.662075] device-mapper: uevent: version 1.0.3 [ 30.662214] device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: dm-devel@redhat.com [ 31.611749] wmi: Mapper loaded [ 31.692565] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 31.706413] ipmi message handler version 39.2 [ 31.733964] ioatdma: Intel(R) QuickData Technology Driver 4.00 [ 31.741550] ipmi device interface [ 31.741648] ioatdma 0000:00:04.0: irq 182 for MSI/MSI-X [ 31.742144] igb 0000:02:00.0: DCA enabled [ 31.742158] igb 0000:02:00.1: DCA enabled [ 31.742350] ioatdma 0000:00:04.1: irq 184 for MSI/MSI-X [ 31.742762] igb 0000:02:00.2: DCA enabled [ 31.742777] igb 0000:02:00.3: DCA enabled [ 31.742936] ioatdma 0000:00:04.2: irq 185 for MSI/MSI-X [ 31.769148] ioatdma 0000:00:04.3: irq 186 for MSI/MSI-X [ 31.769756] ioatdma 0000:00:04.4: irq 187 for MSI/MSI-X [ 31.770319] ioatdma 0000:00:04.5: irq 188 for MSI/MSI-X [ 31.770875] ioatdma 0000:00:04.6: irq 189 for MSI/MSI-X [ 31.771440] ioatdma 0000:00:04.7: irq 190 for MSI/MSI-X [ 31.772156] ioatdma 0000:80:04.0: irq 192 for MSI/MSI-X [ 31.772772] ioatdma 0000:80:04.1: irq 194 for MSI/MSI-X [ 31.773333] ioatdma 0000:80:04.2: irq 195 for MSI/MSI-X [ 31.773885] ioatdma 0000:80:04.3: irq 196 for MSI/MSI-X [ 31.774406] ioatdma 0000:80:04.4: irq 197 for MSI/MSI-X [ 31.774927] ioatdma 0000:80:04.5: irq 198 for MSI/MSI-X [ 31.775452] ioatdma 0000:80:04.6: irq 199 for MSI/MSI-X [ 31.775987] ioatdma 0000:80:04.7: irq 200 for MSI/MSI-X [ 31.902690] ACPI Warning: SystemIO range 0x0000000000004000-0x000000000000401f conflicts with OpRegion 0x0000000000004000-0x000000000000400f (\_SB_.PCI0.SBUS.SMBI) (20130517/utaddress-254) [ 31.924013] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver [ 31.936835] i801_smbus 0000:07:00.3: Enabling SMBus device [ 31.944240] i801_smbus 0000:07:00.3: SMBus using PCI interrupt [ 31.970056] IPMI System Interface driver. [ 31.976066] ipmi_si: probing via SMBIOS [ 31.981666] ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0 [ 31.989601] ipmi_si: Adding SMBIOS-specified kcs state machine [ 31.997463] ipmi_si: probing via SPMI [ 32.002852] ipmi_si: SPMI: io 0xca2 regsize 1 spacing 1 irq 0 [ 32.010571] ipmi_si: Adding SPMI-specified kcs state machine duplicate interface [ 32.020218] ipmi_si: Trying SMBIOS-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0 [ 32.033171] sd 1:0:0:0: Attached scsi generic sg0 type 0 [ 32.040000] sd 1:0:1:0: Attached scsi generic sg1 type 0 [ 32.046865] sd 0:0:0:0: Attached scsi generic sg2 type 0 [ 32.053700] sd 0:0:1:0: Attached scsi generic sg3 type 0 [ 32.053733] mei_me 0000:00:16.0: Device doesn't have valid ME Interface [ 32.068716] sd 0:0:2:0: Attached scsi generic sg4 type 0 [ 32.075547] sd 0:0:3:0: Attached scsi generic sg5 type 0 [ 32.082381] sd 0:0:4:0: Attached scsi generic sg6 type 0 [ 32.089158] sd 0:0:5:0: Attached scsi generic sg7 type 0 [ 32.095957] sd 0:0:6:0: Attached scsi generic sg8 type 0 [ 32.102755] sd 0:0:7:0: Attached scsi generic sg9 type 0 [ 32.109546] sd 0:0:8:0: Attached scsi generic sg10 type 0 [ 32.116399] sd 0:0:9:0: Attached scsi generic sg11 type 0 [ 32.123248] sd 0:0:10:0: Attached scsi generic sg12 type 0 [ 32.127047] ipmi_si ipmi_si.0: Found new BMC (man_id: 0x000157, prod_id: 0x0049, dev_id: 0x21) [ 32.127119] ipmi_si ipmi_si.0: IPMI kcs interface initialized [ 32.147632] sd 0:0:11:0: Attached scsi generic sg13 type 0 [ 32.154851] sd 0:0:12:0: Attached scsi generic sg14 type 0 [ 32.161957] sd 0:0:13:0: Attached scsi generic sg15 type 0 [ 32.162129] IPMI SSIF Interface driver [ 32.173896] sd 0:0:14:0: Attached scsi generic sg16 type 0 [ 32.181261] sd 0:0:15:0: Attached scsi generic sg17 type 0 [ 32.188328] sd 0:0:16:0: Attached scsi generic sg18 type 0 [ 32.195823] sd 0:0:17:0: Attached scsi generic sg19 type 0 [ 32.203175] sd 0:0:18:0: Attached scsi generic sg20 type 0 [ 32.210263] sd 0:0:19:0: Attached scsi generic sg21 type 0 [ 32.217566] sd 0:0:20:0: Attached scsi generic sg22 type 0 [ 32.224744] scsi 0:0:21:0: Attached scsi generic sg23 type 13 [ 32.231992] sd 0:0:22:0: Attached scsi generic sg24 type 0 [ 32.238915] sd 0:0:23:0: Attached scsi generic sg25 type 0 [ 32.245880] sd 0:0:24:0: Attached scsi generic sg26 type 0 [ 32.252918] sd 0:0:25:0: Attached scsi generic sg27 type 0 [ 32.260005] sd 0:0:26:0: Attached scsi generic sg28 type 0 [ 32.266889] sd 0:0:27:0: Attached scsi generic sg29 type 0 [ 32.273685] sd 0:0:28:0: Attached scsi generic sg30 type 0 [ 32.280586] sd 0:0:29:0: Attached scsi generic sg31 type 0 [ 32.287778] sd 0:0:30:0: Attached scsi generic sg32 type 0 [ 32.294863] sd 0:0:31:0: Attached scsi generic sg33 type 0 [ 32.301592] sd 0:0:32:0: Attached scsi generic sg34 type 0 [ 32.308316] sd 0:0:33:0: Attached scsi generic sg35 type 0 [ 32.315223] sd 0:0:34:0: Attached scsi generic sg36 type 0 [ 32.322158] sd 0:0:35:0: Attached scsi generic sg37 type 0 [ 32.329070] sd 0:0:36:0: Attached scsi generic sg38 type 0 [ 32.335894] sd 0:0:37:0: Attached scsi generic sg39 type 0 [ 32.342592] sd 0:0:38:0: Attached scsi generic sg40 type 0 [ 32.349178] sd 0:0:39:0: Attached scsi generic sg41 type 0 [ 32.355813] sd 0:0:40:0: Attached scsi generic sg42 type 0 [ 32.362738] sd 0:0:41:0: Attached scsi generic sg43 type 0 [ 32.369711] sd 0:0:42:0: Attached scsi generic sg44 type 0 [ 32.376386] sd 0:0:43:0: Attached scsi generic sg45 type 0 [ 32.383307] sd 0:0:44:0: Attached scsi generic sg46 type 0 [ 32.394110] sd 0:0:45:0: Attached scsi generic sg47 type 0 [ 32.402594] scsi 0:0:46:0: Attached scsi generic sg48 type 13 [ 32.411849] sd 8:0:0:0: Attached scsi generic sg49 type 0 [ 32.418782] sd 8:0:1:0: Attached scsi generic sg50 type 0 [ 32.425744] sd 8:0:2:0: Attached scsi generic sg51 type 0 [ 32.432532] sd 8:0:3:0: Attached scsi generic sg52 type 0 [ 32.438948] sd 8:0:4:0: Attached scsi generic sg53 type 0 [ 32.445697] sd 8:0:5:0: Attached scsi generic sg54 type 0 [ 32.452463] sd 8:0:6:0: Attached scsi generic sg55 type 0 [ 32.460398] sd 8:0:7:0: Attached scsi generic sg56 type 0 [ 32.466864] sd 8:0:8:0: Attached scsi generic sg57 type 0 [ 32.473212] sd 8:0:9:0: Attached scsi generic sg58 type 0 [ 32.479540] sd 8:0:10:0: Attached scsi generic sg59 type 0 [ 32.485965] sd 8:0:11:0: Attached scsi generic sg60 type 0 [ 32.492381] sd 8:0:12:0: Attached scsi generic sg61 type 0 [ 32.498805] sd 8:0:13:0: Attached scsi generic sg62 type 0 [ 32.505220] sd 8:0:14:0: Attached scsi generic sg63 type 0 [ 32.511624] sd 8:0:15:0: Attached scsi generic sg64 type 0 [ 32.518029] sd 8:0:16:0: Attached scsi generic sg65 type 0 [ 32.524431] sd 8:0:17:0: Attached scsi generic sg66 type 0 [ 32.530836] sd 8:0:18:0: Attached scsi generic sg67 type 0 [ 32.537234] sd 8:0:19:0: Attached scsi generic sg68 type 0 [ 32.543633] sd 8:0:20:0: Attached scsi generic sg69 type 0 [ 32.550031] scsi 8:0:21:0: Attached scsi generic sg70 type 13 [ 32.556715] sd 8:0:22:0: Attached scsi generic sg71 type 0 [ 32.563113] sd 8:0:23:0: Attached scsi generic sg72 type 0 [ 32.569509] sd 8:0:24:0: Attached scsi generic sg73 type 0 [ 32.575905] sd 8:0:25:0: Attached scsi generic sg74 type 0 [ 32.582299] sd 8:0:26:0: Attached scsi generic sg75 type 0 [ 32.588700] sd 8:0:27:0: Attached scsi generic sg76 type 0 [ 32.595095] sd 8:0:28:0: Attached scsi generic sg77 type 0 [ 32.601592] sd 8:0:29:0: Attached scsi generic sg78 type 0 [ 32.608017] sd 8:0:30:0: Attached scsi generic sg79 type 0 [ 32.614572] sd 8:0:31:0: Attached scsi generic sg80 type 0 [ 32.621269] sd 8:0:32:0: Attached scsi generic sg81 type 0 [ 32.627899] sd 8:0:33:0: Attached scsi generic sg82 type 0 [ 32.634515] sd 8:0:34:0: Attached scsi generic sg83 type 0 [ 32.634713] device-mapper: multipath round-robin: version 1.2.0 loaded [ 32.648780] sd 8:0:35:0: Attached scsi generic sg84 type 0 [ 32.655629] sd 8:0:36:0: Attached scsi generic sg85 type 0 [ 32.662331] sd 8:0:37:0: Attached scsi generic sg86 type 0 [ 32.669099] sd 8:0:38:0: Attached scsi generic sg87 type 0 [ 32.675759] sd 8:0:39:0: Attached scsi generic sg88 type 0 [ 32.682515] sd 8:0:40:0: Attached scsi generic sg89 type 0 [ 32.689171] sd 8:0:41:0: Attached scsi generic sg90 type 0 [ 32.696004] sd 8:0:42:0: Attached scsi generic sg91 type 0 [ 32.702932] sd 8:0:43:0: Attached scsi generic sg92 type 0 [ 32.703217] sd 8:0:44:0: Attached scsi generic sg93 type 0 [ 32.703540] sd 8:0:45:0: Attached scsi generic sg94 type 0 [ 32.703958] scsi 8:0:46:0: Attached scsi generic sg95 type 13 [ 32.750821] input: PC Speaker as /devices/platform/pcspkr/input/input3 [ 32.787561] iTCO_vendor_support: vendor-support=0 [ 32.789978] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11 [ 32.790069] iTCO_wdt: unable to reset NO_REBOOT flag, device disabled by hardware/BIOS [ 33.248012] device-mapper: table: 253:91: multipath: error getting device [ 33.256221] device-mapper: ioctl: error adding target to table [ 33.344592] ses 0:0:21:0: Attached Enclosure device [ 33.350988] ses 0:0:46:0: Attached Enclosure device [ 33.357141] ses 8:0:21:0: Attached Enclosure device [ 33.363292] ses 8:0:46:0: Attached Enclosure device [ 33.422139] Adding 16319484k swap on /dev/sda2. Priority:-1 extents:1 across:16319484k FS [ 33.479934] AES CTR mode by8 optimization enabled [ 33.483821] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni) [ 33.565679] alg: No test for crc32 (crc32-pclmul) [ 33.675505] SPL: Loaded module v0.7.3-1 [ 33.800377] znvpair: module license 'CDDL' taints kernel. [ 33.800378] Disabling lock debugging due to kernel taint [ 33.824977] type=1305 audit(1510797712.557:3): audit_pid=2379 old=0 auid=4294967295 ses=4294967295 res=1 [ 33.911880] kvm: disabled by bios [ 34.434939] intel_rapl: Found RAPL domain package [ 34.440929] intel_rapl: Found RAPL domain core [ 34.446578] intel_rapl: Found RAPL domain dram [ 34.452257] intel_rapl: Found RAPL domain package [ 34.458186] intel_rapl: Found RAPL domain core [ 34.463801] intel_rapl: Found RAPL domain dram [ 35.099125] EDAC MC: Ver: 3.0.0 [ 35.234152] EDAC sbridge: Seeking for: PCI ID 8086:3ca0 [ 35.234171] EDAC sbridge: Seeking for: PCI ID 8086:3ca0 [ 35.234178] EDAC sbridge: Seeking for: PCI ID 8086:3ca0 [ 35.234182] EDAC sbridge: Seeking for: PCI ID 8086:3ca8 [ 35.234186] EDAC sbridge: Seeking for: PCI ID 8086:3ca8 [ 35.234190] EDAC sbridge: Seeking for: PCI ID 8086:3ca8 [ 35.234191] EDAC sbridge: Seeking for: PCI ID 8086:3c71 [ 35.234195] EDAC sbridge: Seeking for: PCI ID 8086:3c71 [ 35.234199] EDAC sbridge: Seeking for: PCI ID 8086:3c71 [ 35.234201] EDAC sbridge: Seeking for: PCI ID 8086:3caa [ 35.234205] EDAC sbridge: Seeking for: PCI ID 8086:3caa [ 35.234209] EDAC sbridge: Seeking for: PCI ID 8086:3caa [ 35.234210] EDAC sbridge: Seeking for: PCI ID 8086:3cab [ 35.234214] EDAC sbridge: Seeking for: PCI ID 8086:3cab [ 35.234218] EDAC sbridge: Seeking for: PCI ID 8086:3cab [ 35.234219] EDAC sbridge: Seeking for: PCI ID 8086:3cac [ 35.234223] EDAC sbridge: Seeking for: PCI ID 8086:3cac [ 35.234227] EDAC sbridge: Seeking for: PCI ID 8086:3cac [ 35.234228] EDAC sbridge: Seeking for: PCI ID 8086:3cad [ 35.234233] EDAC sbridge: Seeking for: PCI ID 8086:3cad [ 35.234237] EDAC sbridge: Seeking for: PCI ID 8086:3cad [ 35.234238] EDAC sbridge: Seeking for: PCI ID 8086:3cb8 [ 35.234243] EDAC sbridge: Seeking for: PCI ID 8086:3cb8 [ 35.234246] EDAC sbridge: Seeking for: PCI ID 8086:3cb8 [ 35.234247] EDAC sbridge: Seeking for: PCI ID 8086:3cf4 [ 35.234250] EDAC sbridge: Seeking for: PCI ID 8086:3cf4 [ 35.234254] EDAC sbridge: Seeking for: PCI ID 8086:3cf4 [ 35.234256] EDAC sbridge: Seeking for: PCI ID 8086:3cf6 [ 35.234260] EDAC sbridge: Seeking for: PCI ID 8086:3cf6 [ 35.234264] EDAC sbridge: Seeking for: PCI ID 8086:3cf6 [ 35.234266] EDAC sbridge: Seeking for: PCI ID 8086:3cf5 [ 35.234270] EDAC sbridge: Seeking for: PCI ID 8086:3cf5 [ 35.234273] EDAC sbridge: Seeking for: PCI ID 8086:3cf5 [ 35.234430] EDAC MC0: Giving out device to 'sbridge_edac.c' 'Sandy Bridge Socket#0': DEV 0000:7f:0e.0 [ 35.234553] EDAC MC1: Giving out device to 'sbridge_edac.c' 'Sandy Bridge Socket#1': DEV 0000:ff:0e.0 [ 35.234553] EDAC sbridge: Ver: 1.1.1 [ 36.351651] ZFS: Loaded module v0.7.3-1, ZFS pool version 5000, ZFS filesystem version 5 [ 36.736096] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 36.825264] igb 0000:02:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX [ 36.826753] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready [ 36.831935] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready [ 36.913767] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready [ 36.921899] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 36.938316] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready [ 37.057911] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready [ 37.073025] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready [ 37.183792] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready [ 38.393138] mlx4_en: Mellanox ConnectX HCA Ethernet driver v4.2-1.0.0 [ 38.760515] card: mlx4_0, QP: 0x220, inline size: 120 [ 38.783530] card: mlx4_0, QP: 0x300, inline size: 120 [ 38.816908] ib0: enabling connected mode will cause multicast packet drops [ 38.817129] ib0: mtu > 4092 will cause multicast packet drops. [ 39.635908] IPv6: ADDRCONF(NETDEV_UP): ib0: link is not ready [ 39.664451] IPv6: ADDRCONF(NETDEV_UP): ib0: link is not ready [ 39.676379] IPv6: ADDRCONF(NETDEV_UP): ib1: link is not ready [ 39.702761] IPv6: ADDRCONF(NETDEV_UP): ib1: link is not ready [ 46.390042] IPv6: ADDRCONF(NETDEV_UP): ib0: link is not ready [ 46.449013] IPv6: ADDRCONF(NETDEV_UP): ib0: link is not ready [ 46.459286] IPv6: ADDRCONF(NETDEV_UP): ib0: link is not ready [ 46.692185] FS-Cache: Loaded [ 46.711453] FS-Cache: Netfs 'nfs' registered for caching [ 46.715231] Key type dns_resolver registered [ 46.742930] NFS: Registering the id_resolver key type [ 46.742934] Key type id_resolver registered [ 46.742934] Key type id_legacy registered [ 59.679047] IPv6: ADDRCONF(NETDEV_CHANGE): ib0: link becomes ready [ 2519.898256] LNet: HW NUMA nodes: 2, HW CPU cores: 32, npartitions: 2 [ 2519.908072] alg: No test for adler32 (adler32-zlib) [ 2519.913795] alg: No test for crc32 (crc32-table) [ 2520.746779] Lustre: Lustre: Build Version: 2.10.55 [ 2520.915069] LNet: Added LNI 192.168.1.107@o2ib [8/256/0/180] [ 2526.902115] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 4297193 seconds [ 2527.902067] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4297194 seconds [ 2576.899092] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.109@o2ib: 5 seconds [ 2577.057987] Lustre: soaked-OST0005: new disk, initializing [ 2577.064279] Lustre: srv-soaked-OST0005: No data found on store. Initialize space [ 2590.898240] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4297257 seconds [ 2627.896014] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.109@o2ib: 0 seconds [ 2627.910885] Lustre: soaked-OST0005: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [ 2635.385320] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [ 2653.894318] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4297320 seconds [ 2653.906472] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [ 2691.891999] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 1 seconds [ 2766.887433] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4297433 seconds [ 2766.923012] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [ 2893.879681] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4297560 seconds [ 2893.891815] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [ 2934.625434] Lustre: soaked-OST000b: new disk, initializing [ 2934.631733] Lustre: srv-soaked-OST000b: No data found on store. Initialize space [ 2993.875809] Lustre: soaked-OST000b: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [ 2999.776844] Lustre: soaked-OST000b: Connection restored to (at 192.168.1.108@o2ib) [ 3057.869693] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 1 seconds [ 3057.881248] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [ 3301.820407] Lustre: soaked-OST0011: new disk, initializing [ 3301.826712] Lustre: srv-soaked-OST0011: No data found on store. Initialize space [ 3322.853544] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4297989 seconds [ 3322.865687] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [ 3359.949477] Lustre: soaked-OST0011: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [ 3368.110606] Lustre: soaked-OST0011: Connection restored to (at 192.168.1.108@o2ib) [ 3667.537218] Lustre: soaked-OST0017: new disk, initializing [ 3667.555446] Lustre: srv-soaked-OST0017: No data found on store. Initialize space [ 3725.831562] Lustre: soaked-OST0017: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [ 3733.982176] Lustre: soaked-OST0017: Connection restored to (at 192.168.1.108@o2ib) [ 3834.733653] Lustre: soaked-OST0005: Connection restored to ea9e99b5-0c32-a02b-c8e9-0e0af8375b0d (at 192.168.1.116@o2ib) [ 3865.820458] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4298532 seconds [ 3865.949637] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 12 previous similar messages [ 3943.066670] Lustre: soaked-OST0005: Connection restored to a532d19f-347c-0eff-e707-eca0f50b6da1 (at 192.168.1.136@o2ib) [ 3943.078816] Lustre: Skipped 3 previous similar messages [ 4051.454252] Lustre: soaked-OST0005: Connection restored to 0362cadb-53db-4b8d-6d66-d0420a70993b (at 192.168.1.137@o2ib) [ 4051.492316] Lustre: Skipped 3 previous similar messages [ 4160.175084] Lustre: soaked-OST0005: Connection restored to 0f4bee86-d847-4849-871c-999728a1d5eb (at 192.168.1.138@o2ib) [ 4160.175087] Lustre: soaked-OST000b: Connection restored to 0f4bee86-d847-4849-871c-999728a1d5eb (at 192.168.1.138@o2ib) [ 4160.438419] Lustre: Skipped 2 previous similar messages [ 4377.252927] Lustre: soaked-OST0005: Connection restored to 9972e6fb-d5a8-54ee-8435-a4d933d3b728 (at 192.168.1.140@o2ib) [ 4377.384683] Lustre: Skipped 7 previous similar messages [ 4521.780391] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 3 seconds [ 4521.816840] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 8 previous similar messages [ 4703.087343] Lustre: soaked-OST0005: Connection restored to b1010b82-5169-87e5-3f1a-977d836fd667 (at 192.168.1.143@o2ib) [ 4703.102138] Lustre: Skipped 10 previous similar messages [ 5127.743446] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 6 seconds [ 5127.757693] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 7 previous similar messages [ 5796.702626] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [ 5796.739169] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [ 6298.088528] Lustre: soaked-OST0005: Connection restored to 10a174c2-51d2-cf71-eee8-f3b18990cb30 (at 192.168.1.119@o2ib) [ 6298.103431] Lustre: Skipped 11 previous similar messages [ 6407.359896] Lustre: soaked-OST000b: Connection restored to cdfa6078-dd66-cd68-ba8a-e51adc0b7463 (at 192.168.1.120@o2ib) [ 6407.398274] Lustre: Skipped 3 previous similar messages [ 6625.346773] Lustre: soaked-OST0005: Connection restored to f360cbdf-c189-1f9b-11ce-46bac06079cb (at 192.168.1.122@o2ib) [ 6625.385130] Lustre: Skipped 7 previous similar messages [ 6655.650268] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [ 6655.686912] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 5 previous similar messages [ 6951.502347] Lustre: soaked-OST0005: Connection restored to 2f7ec30e-b35a-b8a2-0749-17e19834672c (at 192.168.1.125@o2ib) [ 6951.634173] Lustre: Skipped 11 previous similar messages [ 7261.613337] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [ 7261.627772] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [ 7495.379173] Lustre: soaked-OST0005: Connection restored to 4198f89d-9ee1-0ee3-5def-a9fce63497a8 (at 192.168.1.132@o2ib) [ 7495.379175] Lustre: soaked-OST000b: Connection restored to 4198f89d-9ee1-0ee3-5def-a9fce63497a8 (at 192.168.1.132@o2ib) [ 7495.379180] Lustre: Skipped 15 previous similar messages [ 7495.416774] Lustre: Skipped 2 previous similar messages [ 8119.560975] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [ 8119.597658] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [ 8789.520106] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 3 seconds [ 8789.645868] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [ 9521.475483] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [ 9521.512112] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [10253.430816] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 2 seconds [10253.445086] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 5 previous similar messages [11048.382361] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [11048.396597] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [11718.341473] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [11718.377974] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [12387.300665] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [12387.426173] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [13182.252191] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [13182.377658] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [13851.211394] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [13851.225413] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [14646.162921] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [14646.176879] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [15379.118211] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [15379.243541] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [15575.065538] Lustre: soaked-OST0005: Connection restored to ed8fa4bb-d36f-0b40-48fa-fe78100782a5 (at 192.168.1.133@o2ib) [15575.065541] Lustre: soaked-OST000b: Connection restored to ed8fa4bb-d36f-0b40-48fa-fe78100782a5 (at 192.168.1.133@o2ib) [15575.328571] Lustre: Skipped 2 previous similar messages [15646.105324] Lustre: soaked-OST0005: Connection restored to febd8698-66cc-4265-6a2c-eec9a1c90d86 (at 192.168.1.134@o2ib) [15646.119822] Lustre: Skipped 3 previous similar messages [15754.690569] Lustre: soaked-OST0005: Connection restored to 8b0c50cd-9f3a-6240-b7ca-f83ce30a2cb0 (at 192.168.1.128@o2ib) [15754.728521] Lustre: Skipped 3 previous similar messages [15863.321671] Lustre: soaked-OST000b: Connection restored to 7138a171-b124-888f-d923-e084f7df0c22 (at 192.168.1.129@o2ib) [15863.453272] Lustre: Skipped 3 previous similar messages [15971.916218] Lustre: soaked-OST0005: Connection restored to d4c5c945-5827-ce7a-8968-7a952a9acfc5 (at 192.168.1.135@o2ib) [15971.930819] Lustre: Skipped 3 previous similar messages [16111.073572] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [16111.198920] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [17134.011128] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [17134.024234] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [17798.732629] Lustre: 21488:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510815471/real 0] req@ffff8808185ef200 x1584188852462464/t0(0) o400->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510815478 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [17798.732643] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [17798.732646] Lustre: soaked-MDT0000-lwp-OST0005: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [17798.943983] Lustre: 21488:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [17804.978247] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510815478/real 0] req@ffff8808185e9e00 x1584188852462560/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510815484 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [17829.976685] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510815503/real 0] req@ffff8808185ee900 x1584188852462624/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510815509 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [17830.074044] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [17877.965863] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.109@o2ib: 4 seconds [17898.972982] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510815578/real 1510815578] req@ffff8808185e8600 x1584188852462688/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510815589 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [17899.077783] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [17934.970361] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510815603/real 0] req@ffff8808185eb600 x1584188852462800/t0(0) o38->soaked-MDT0000-lwp-OST0017@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510815614 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [17935.071659] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [17999.433177] Lustre: soaked-OST000b: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 229 seconds. I think it's dead, and I am evicting it. exp ffff880817fed000, cur 1510815679 expire 1510815529 last 1510815450 [18000.465552] Lustre: soaked-OST0005: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff880814fc0000, cur 1510815680 expire 1510815530 last 1510815453 [18000.542402] Lustre: Skipped 1 previous similar message [18014.965468] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510815678/real 0] req@ffff8808185ec800 x1584188852462848/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510815694 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [18015.067961] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [18037.061077] Lustre: soaked-OST0005: Connection restored to soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) [18037.097034] Lustre: Skipped 7 previous similar messages [18073.962533] Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib_1) after server handle changed from 0x6a1009ca5a697cbb to 0xf8a25437a80fac19 [18074.011223] Lustre: MGC192.168.1.108@o2ib: Connection restored to MGC192.168.1.108@o2ib_1 (at 192.168.1.109@o2ib) [18123.959407] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [18124.013961] Lustre: soaked-MDT0000-lwp-OST0017: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [18124.051451] Lustre: Skipped 3 previous similar messages [18153.943287] Lustre: soaked-OST0005: deleting orphan objects from 0x0:440569 to 0x0:449073 [18153.943301] Lustre: soaked-OST000b: deleting orphan objects from 0x0:440427 to 0x0:448945 [18153.943482] Lustre: soaked-OST0017: deleting orphan objects from 0x0:440493 to 0x0:449697 [18153.943495] Lustre: soaked-OST0011: deleting orphan objects from 0x0:440216 to 0x0:446001 [18180.956347] Lustre: 21500:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510815853/real 1510815853] req@ffff8808185eaa00 x1584188852555056/t0(0) o400->soaked-MDT0000-lwp-OST0017@192.168.1.109@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510815860 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [18180.956367] Lustre: soaked-MDT0000-lwp-OST0005: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [18180.956370] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.109@o2ib) was lost; in progress operations using this service will fail [18180.956371] Lustre: Skipped 3 previous similar messages [18181.197961] Lustre: 21500:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 8 previous similar messages [18206.232427] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [18206.285198] LustreError: Skipped 3 previous similar messages [18206.307672] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xf8a25437a80fac19 to 0x6b2dc99032cce64a [18206.308870] Lustre: soaked-MDT0000-lwp-OST000b: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [18206.308873] Lustre: Skipped 4 previous similar messages [18211.803751] Lustre: soaked-OST0005: deleting orphan objects from 0x0:449665 to 0x0:449681 [18211.803753] Lustre: soaked-OST0011: deleting orphan objects from 0x0:446637 to 0x0:446673 [18211.803759] Lustre: soaked-OST000b: deleting orphan objects from 0x0:449565 to 0x0:449585 [18211.803766] Lustre: soaked-OST0017: deleting orphan objects from 0x0:450310 to 0x0:450337 [19330.249456] perf: interrupt took too long (2504 > 2500), lowering kernel.perf_event_max_sample_rate to 79000 [20150.827119] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [20150.951325] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 8 previous similar messages [20561.108435] perf: interrupt took too long (3157 > 3130), lowering kernel.perf_event_max_sample_rate to 63000 [21013.068648] Lustre: 21495:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510818686/real 0] req@ffff880804f9d400 x1584188917547696/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510818693 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [21013.109374] Lustre: 21495:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 9 previous similar messages [21013.123500] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [21019.144212] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510818693/real 0] req@ffff880804f99b00 x1584188917547776/t0(0) o250->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 520/544 e 0 to 1 dl 1510818699 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [21031.215249] Lustre: 21499:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510818686/real 1510818711] req@ffff880804f99200 x1584188917547760/t0(0) o400->soaked-MDT0000-lwp-OST0017@192.168.1.108@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510818723 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [21031.215263] Lustre: soaked-MDT0000-lwp-OST0011: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [21031.215265] Lustre: Skipped 4 previous similar messages [21031.286608] Lustre: 21499:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 8 previous similar messages [21062.213583] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510818736/real 1510818736] req@ffff880804f98c00 x1584188917547904/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510818742 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [21117.210263] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510818786/real 1510818786] req@ffff880804f9f500 x1584188917548064/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510818797 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [21117.252134] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 9 previous similar messages [21221.925369] Lustre: soaked-OST0017: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 229 seconds. I think it's dead, and I am evicting it. exp ffff880265050c00, cur 1510818901 expire 1510818751 last 1510818672 [21222.002319] Lustre: Skipped 1 previous similar message [21226.137761] Lustre: soaked-OST000b: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 231 seconds. I think it's dead, and I am evicting it. exp ffff880754300c00, cur 1510818906 expire 1510818756 last 1510818675 [21226.214761] Lustre: Skipped 1 previous similar message [21227.203536] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510818886/real 1510818886] req@ffff880804f9bf00 x1584188917548416/t0(0) o38->soaked-MDT0000-lwp-OST0017@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510818907 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [21227.308376] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 19 previous similar messages [21253.922982] Lustre: soaked-OST0005: Connection restored to soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) [21253.959291] Lustre: Skipped 7 previous similar messages [21285.758005] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 5 seconds [21299.757115] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 4 seconds [21299.794268] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [21306.199300] LustreError: 167-0: soaked-MDT0000-lwp-OST000b: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [21306.248704] LustreError: Skipped 3 previous similar messages [21306.271256] Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib_1) after server handle changed from 0x6b2dc99032cce64a to 0xf8a25437a839b60d [21306.272082] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [21318.836546] Lustre: soaked-OST0005: deleting orphan objects from 0x0:1974961 to 0x0:1989361 [21318.836548] Lustre: soaked-OST000b: deleting orphan objects from 0x0:1972123 to 0x0:1977265 [21318.836557] Lustre: soaked-OST0011: deleting orphan objects from 0x0:1965404 to 0x0:1973857 [21318.836562] Lustre: soaked-OST0017: deleting orphan objects from 0x0:1973714 to 0x0:1978737 [21350.379054] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [21350.407718] Lustre: Skipped 5 previous similar messages [21363.195276] Lustre: 21489:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510819036/real 1510819036] req@ffff880804f98600 x1584188918534512/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.109@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510819043 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [21363.298927] Lustre: 21489:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 9 previous similar messages [21363.334340] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.109@o2ib) was lost; in progress operations using this service will fail [21375.744433] LustreError: 78296:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0017: can't get bonus, rc = -17 [21383.331954] LustreError: 78295:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0011: can't get bonus, rc = -17 [21383.368788] LustreError: 78295:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 91 previous similar messages [21388.386452] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xf8a25437a839b60d to 0xcf45ba53bf4ef140 [21392.193518] Lustre: soaked-MDT0000-lwp-OST0011: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [21392.253110] Lustre: Skipped 4 previous similar messages [21402.606577] Lustre: soaked-OST0005: deleting orphan objects from 0x0:2003571 to 0x0:2003665 [21402.606597] Lustre: soaked-OST000b: deleting orphan objects from 0x0:1991476 to 0x0:1991665 [21402.606599] Lustre: soaked-OST0017: deleting orphan objects from 0x0:1992994 to 0x0:1993137 [21402.606601] Lustre: soaked-OST0011: deleting orphan objects from 0x0:1988065 to 0x0:1988129 [21417.270070] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [21417.319522] LustreError: Skipped 3 previous similar messages [21419.402899] LustreError: 78295:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0011: can't get bonus, rc = -17 [21419.439574] LustreError: 78295:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 106 previous similar messages [22368.541117] perf: interrupt took too long (3966 > 3946), lowering kernel.perf_event_max_sample_rate to 50000 [24085.670033] perf: interrupt took too long (4963 > 4957), lowering kernel.perf_event_max_sample_rate to 40000 [28465.424143] perf: interrupt took too long (6208 > 6203), lowering kernel.perf_event_max_sample_rate to 32000 [36148.404495] Lustre: 21492:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510833822/real 0] req@ffff8807b86d4800 x1584189294112240/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510833829 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [36148.404511] Lustre: soaked-MDT0000-lwp-OST000b: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [36148.404514] Lustre: Skipped 1 previous similar message [36148.583935] Lustre: 21492:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 13 previous similar messages [36148.618844] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [36198.667933] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510833879/real 1510833879] req@ffff8807b86d0f00 x1584189294112512/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510833890 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [36198.772549] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 10 previous similar messages [36289.661907] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510833954/real 1510833954] req@ffff8807b86d5100 x1584189294112752/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510833970 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [36290.014898] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 14 previous similar messages [36322.840874] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 2 seconds [36322.966547] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [36361.432251] Lustre: soaked-OST0005: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff88069f4a3400, cur 1510834042 expire 1510833892 last 1510833815 [36361.693519] Lustre: Skipped 1 previous similar message [36394.578480] Lustre: soaked-OST0005: Connection restored to soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) [36394.699894] Lustre: Skipped 9 previous similar messages [36397.836334] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 12 seconds [36398.656074] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [36398.828735] LustreError: Skipped 3 previous similar messages [36409.835564] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 11 seconds [36423.654282] LustreError: 167-0: soaked-MDT0000-lwp-OST000b: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [36423.825106] Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib_1) after server handle changed from 0xcf45ba53bf4ef140 to 0xf8a25437a8cbf75b [36423.826172] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [36423.826175] Lustre: Skipped 1 previous similar message [36504.446429] Lustre: soaked-OST0011: deleting orphan objects from 0x0:11856510 to 0x0:11864289 [36504.446431] Lustre: soaked-OST0005: deleting orphan objects from 0x0:11951873 to 0x0:11965585 [36504.446432] Lustre: soaked-OST0017: deleting orphan objects from 0x0:11915580 to 0x0:11925921 [36504.446433] Lustre: soaked-OST000b: deleting orphan objects from 0x0:11883519 to 0x0:11890433 [36523.648126] LustreError: 11-0: soaked-MDT0000-lwp-OST0011: operation obd_ping to node 192.168.1.109@o2ib failed: rc = -107 [36523.648139] Lustre: soaked-MDT0000-lwp-OST0017: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [36523.648142] Lustre: Skipped 3 previous similar messages [36523.691431] LustreError: Skipped 3 previous similar messages [36536.479945] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [36536.492329] Lustre: Skipped 4 previous similar messages [36548.646819] LustreError: 167-0: soaked-MDT0000-lwp-OST0011: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [36548.668921] LustreError: Skipped 2 previous similar messages [36555.645676] Lustre: 21501:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510834229/real 1510834229] req@ffff880759429e00 x1584189295453344/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.109@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510834236 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [36555.687013] Lustre: 21501:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 19 previous similar messages [36555.701231] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.109@o2ib) was lost; in progress operations using this service will fail [36577.244644] Lustre: soaked-OST0005: deleting orphan objects from 0x0:11976435 to 0x0:11976497 [36577.244665] Lustre: soaked-OST0017: deleting orphan objects from 0x0:11936858 to 0x0:11936929 [36577.244668] Lustre: soaked-OST000b: deleting orphan objects from 0x0:11901375 to 0x0:11901409 [36577.244670] Lustre: soaked-OST0011: deleting orphan objects from 0x0:11875220 to 0x0:11875297 [36580.721333] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xf8a25437a8cbf75b to 0xff5a154f932fafdf [36580.737488] Lustre: MGC192.168.1.108@o2ib: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [36580.749670] Lustre: Skipped 5 previous similar messages [37104.793172] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [37104.919098] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [37168.789307] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 9 seconds [37336.836036] LustreError: 15897:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0005: can't get bonus, rc = -17 [37345.622415] LustreError: 15896:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST000b: can't get bonus, rc = -17 [37345.751824] LustreError: 15896:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 1 previous similar message [37346.757349] LustreError: 15896:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST000b: can't get bonus, rc = -17 [37346.886623] LustreError: 15896:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 94 previous similar messages [37350.439126] LustreError: 15898:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0011: can't get bonus, rc = -17 [37350.568725] LustreError: 15898:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 1 previous similar message [37355.376829] LustreError: 15898:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0011: can't get bonus, rc = -17 [37355.506161] LustreError: 15898:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 12 previous similar messages [37367.949057] LustreError: 15899:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0017: can't get bonus, rc = -17 [37368.078427] LustreError: 15899:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 59 previous similar messages [40587.483814] Lustre: 21488:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510838261/real 0] req@ffff8805fbc2fb00 x1584189411696576/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510838268 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [40587.522539] Lustre: 21488:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 1 previous similar message [40587.535941] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [40597.483161] Lustre: soaked-MDT0000-lwp-OST0017: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [40597.506604] Lustre: Skipped 5 previous similar messages [40628.512295] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510838303/real 1510838303] req@ffff8805fbc2aa00 x1584189411696784/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510838309 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [40628.552814] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 9 previous similar messages [40697.508474] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510838378/real 1510838378] req@ffff8805fbc2ad00 x1584189411697024/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510838394 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [40697.613120] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 14 previous similar messages [40778.569129] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [40784.048552] Lustre: soaked-OST0017: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff880251067000, cur 1510838465 expire 1510838315 last 1510838238 [40784.309426] Lustre: Skipped 3 previous similar messages [40787.584014] Lustre: soaked-OST0011: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff88068571e400, cur 1510838468 expire 1510838318 last 1510838241 [40853.564613] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 6 seconds [40898.495802] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510838553/real 1510838553] req@ffff8805fbc2c200 x1584189411697424/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510838579 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [40898.536853] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 24 previous similar messages [40948.880871] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [40948.892564] Lustre: Skipped 1 previous similar message [41022.489012] LustreError: 167-0: soaked-MDT0000-lwp-OST0011: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [41022.541053] LustreError: Skipped 3 previous similar messages [41022.563104] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xff5a154f932fafdf to 0x2ddb37b97a08b26a [41022.564172] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [41022.564175] Lustre: Skipped 1 previous similar message [41087.918918] Lustre: soaked-OST0005: deleting orphan objects from 0x0:13876728 to 0x0:13884321 [41087.918988] Lustre: soaked-OST000b: deleting orphan objects from 0x0:13799071 to 0x0:13807649 [41087.919015] Lustre: soaked-OST0017: deleting orphan objects from 0x0:13826998 to 0x0:13837025 [41087.919044] Lustre: soaked-OST0011: deleting orphan objects from 0x0:13771511 to 0x0:13779537 [41504.459843] Lustre: 21502:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510839178/real 0] req@ffff88068b372700 x1584189421082496/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510839185 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [41504.800643] Lustre: 21502:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 14 previous similar messages [41504.921650] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [41521.943194] Lustre: soaked-MDT0000-lwp-OST0017: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [41522.146173] Lustre: Skipped 3 previous similar messages [41624.517527] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 6 seconds [41624.532420] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [41687.513687] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [41687.550799] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [41710.501098] Lustre: soaked-OST0011: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 233 seconds. I think it's dead, and I am evicting it. exp ffff880205076000, cur 1510839391 expire 1510839241 last 1510839158 [41710.577879] Lustre: Skipped 2 previous similar messages [41750.509913] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 5 seconds [41763.509065] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 18 seconds [41775.508383] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 5 seconds [41889.501343] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 19 seconds [41889.628404] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [41890.443691] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [41890.537118] Lustre: Skipped 7 previous similar messages [41922.187176] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [41922.359735] LustreError: Skipped 3 previous similar messages [41922.431673] Lustre: soaked-MDT0000-lwp-OST0017: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [41927.499028] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 5 seconds [41947.185403] LustreError: 167-0: soaked-MDT0000-lwp-OST000b: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [41947.358260] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0x2ddb37b97a08b26a to 0xd61ed27a2c0dd634 [41947.359236] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [41952.525711] LustreError: 11-0: soaked-MDT0000-lwp-OST0017: operation lfsck_query to node 192.168.1.108@o2ib failed: rc = -11 [41977.356254] LustreError: 11-0: soaked-MDT0000-lwp-OST000b: operation lfsck_query to node 192.168.1.108@o2ib failed: rc = -11 [41977.492354] LustreError: Skipped 1 previous similar message [41987.211529] Lustre: soaked-OST0011: deleting orphan objects from 0x0:13968655 to 0x0:13974881 [41987.211531] Lustre: soaked-OST0005: deleting orphan objects from 0x0:14073627 to 0x0:14083649 [41987.211537] Lustre: soaked-OST000b: deleting orphan objects from 0x0:13997391 to 0x0:14007393 [41987.211539] Lustre: soaked-OST0017: deleting orphan objects from 0x0:14025805 to 0x0:14033601 [42279.164626] Lustre: 21474:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510839953/real 0] req@ffff880259334800 x1584189428120256/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510839960 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [42279.203876] Lustre: 21474:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 64 previous similar messages [42279.217683] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [42289.938725] Lustre: soaked-MDT0000-lwp-OST000b: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [42289.962741] Lustre: Skipped 3 previous similar messages [42491.796716] Lustre: soaked-OST000b: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 230 seconds. I think it's dead, and I am evicting it. exp ffff88072c0df000, cur 1510840173 expire 1510840023 last 1510839943 [42491.826544] Lustre: Skipped 3 previous similar messages [42495.859866] Lustre: soaked-OST0005: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 232 seconds. I think it's dead, and I am evicting it. exp ffff8807e6849400, cur 1510840177 expire 1510840027 last 1510839945 [42495.888813] Lustre: Skipped 2 previous similar messages [42545.461380] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 7 seconds [42671.453704] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 8 seconds [42671.579541] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 5 previous similar messages [42738.685630] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [42738.697150] Lustre: Skipped 3 previous similar messages [42739.944101] LustreError: 167-0: soaked-MDT0000-lwp-OST000b: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [42739.965605] LustreError: Skipped 2 previous similar messages [42739.975203] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xd61ed27a2c0dd634 to 0x7e0ad19d5486fc08 [42739.976091] Lustre: soaked-MDT0000-lwp-OST000b: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [42739.976094] Lustre: Skipped 2 previous similar messages [42825.598890] Lustre: soaked-OST000b: deleting orphan objects from 0x0:14126455 to 0x0:14143377 [42825.598891] Lustre: soaked-OST0011: deleting orphan objects from 0x0:14092177 to 0x0:14102369 [42825.598892] Lustre: soaked-OST0005: deleting orphan objects from 0x0:14202741 to 0x0:14219665 [42825.598893] Lustre: soaked-OST0017: deleting orphan objects from 0x0:14152592 to 0x0:14159697 [44246.852601] Lustre: 21485:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510841921/real 0] req@ffff8803efe26600 x1584189434524528/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510841928 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [44247.190711] Lustre: 21485:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 64 previous similar messages [44247.311561] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [44264.164427] Lustre: soaked-MDT0000-lwp-OST0005: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [44264.367044] Lustre: Skipped 3 previous similar messages [44325.426779] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510841995/real 1510841995] req@ffff8803189bfb00 x1584189434524800/t0(0) o38->soaked-MDT0000-lwp-OST0005@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510842006 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [44325.468082] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 19 previous similar messages [44351.351243] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 2 seconds [44351.365926] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [44441.208985] Lustre: soaked-OST0017: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 228 seconds. I think it's dead, and I am evicting it. exp ffff88072769c000, cur 1510842122 expire 1510841972 last 1510841894 [44442.852436] Lustre: soaked-OST0005: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff88082431f000, cur 1510842124 expire 1510841974 last 1510841897 [44442.928631] Lustre: Skipped 1 previous similar message [44446.038166] Lustre: soaked-OST000b: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 229 seconds. I think it's dead, and I am evicting it. exp ffff88042baaa800, cur 1510842127 expire 1510841977 last 1510841898 [44478.244662] Lustre: soaked-OST0005: Connection restored to soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) [44478.280591] Lustre: Skipped 7 previous similar messages [44489.342865] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 1 seconds [44489.417481] LustreError: 167-0: soaked-MDT0000-lwp-OST0005: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [44489.469814] LustreError: Skipped 3 previous similar messages [44489.493152] Lustre: soaked-MDT0000-lwp-OST0005: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [44490.491725] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510842145/real 0] req@ffff8803189bc200 x1584189434525296/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510842171 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [44490.593546] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 25 previous similar messages [44514.415905] LustreError: 167-0: soaked-MDT0000-lwp-OST0011: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [44514.468764] Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib_1) after server handle changed from 0x7e0ad19d5486fc08 to 0xf8a25437a9992b05 [44514.469988] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [44519.196072] Lustre: soaked-OST0017: deleting orphan objects from 0x0:14261255 to 0x0:14267025 [44519.196075] Lustre: soaked-OST000b: deleting orphan objects from 0x0:14240820 to 0x0:14253793 [44519.196078] Lustre: soaked-OST0005: deleting orphan objects from 0x0:14316507 to 0x0:14321537 [44519.196086] Lustre: soaked-OST0011: deleting orphan objects from 0x0:14203276 to 0x0:14209361 [44539.414332] LustreError: 11-0: soaked-MDT0000-lwp-OST0005: operation obd_ping to node 192.168.1.109@o2ib failed: rc = -107 [44539.414347] Lustre: soaked-MDT0000-lwp-OST000b: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [44539.414349] Lustre: Skipped 1 previous similar message [44539.535692] LustreError: Skipped 4 previous similar messages [44548.921399] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [44548.948213] Lustre: Skipped 5 previous similar messages [44564.412566] LustreError: 167-0: soaked-MDT0000-lwp-OST0011: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [44564.465057] LustreError: Skipped 2 previous similar messages [44564.488499] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [44564.525530] Lustre: Skipped 3 previous similar messages [44571.411812] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.109@o2ib) was lost; in progress operations using this service will fail [44596.463804] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xf8a25437a9992b05 to 0x252512abd1a33332 [44596.510504] Lustre: MGC192.168.1.108@o2ib: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [44610.490753] Lustre: soaked-OST0011: deleting orphan objects from 0x0:14209619 to 0x0:14210385 [44610.490755] Lustre: soaked-OST0017: deleting orphan objects from 0x0:14267538 to 0x0:14267793 [44610.490975] Lustre: soaked-OST000b: deleting orphan objects from 0x0:14253885 to 0x0:14254049 [44610.491101] Lustre: soaked-OST0005: deleting orphan objects from 0x0:14322594 to 0x0:14323105 [44654.332745] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 6 seconds [44654.368041] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [46928.326011] Lustre: 21479:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510844602/real 0] req@ffff8803faeb5400 x1584189479248272/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510844609 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [46928.423095] Lustre: 21479:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 5 previous similar messages [46928.458373] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [46943.471024] Lustre: soaked-MDT0000-lwp-OST0005: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [46943.529125] Lustre: Skipped 5 previous similar messages [46968.546089] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510844650/real 1510844650] req@ffff8803a9696f00 x1584189479248464/t0(0) o38->soaked-MDT0000-lwp-OST0005@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510844661 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [46968.650130] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 9 previous similar messages [47059.540029] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510844725/real 1510844725] req@ffff8803a9696600 x1584189479248704/t0(0) o38->soaked-MDT0000-lwp-OST0005@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510844741 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [47059.894469] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 18 previous similar messages [47142.785675] Lustre: soaked-OST0011: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8805b992a400, cur 1510844824 expire 1510844674 last 1510844597 [47144.125182] Lustre: soaked-OST0017: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8808019adc00, cur 1510844825 expire 1510844675 last 1510844598 [47144.154741] Lustre: Skipped 1 previous similar message [47145.677437] Lustre: soaked-OST0005: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff880667375800, cur 1510844827 expire 1510844677 last 1510844600 [47179.784698] Lustre: soaked-OST0005: Connection restored to soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) [47179.799310] Lustre: Skipped 1 previous similar message [47204.177232] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 11 seconds [47218.531149] LustreError: 167-0: soaked-MDT0000-lwp-OST0011: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [47218.553368] LustreError: Skipped 3 previous similar messages [47218.563138] Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib_1) after server handle changed from 0x252512abd1a33332 to 0xf8a25437a9cd4c5b [47218.564226] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [47218.564229] Lustre: Skipped 2 previous similar messages [47224.606992] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510844875/real 0] req@ffff8803a9697500 x1584189479314400/t0(0) o38->soaked-MDT0000-lwp-OST0005@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510844906 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [47224.648469] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 20 previous similar messages [47242.174951] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 6 seconds [47242.189865] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [47268.527909] LustreError: 167-0: soaked-MDT0000-lwp-OST0005: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [47268.550224] LustreError: Skipped 2 previous similar messages [47268.561319] Lustre: soaked-MDT0000-lwp-OST0005: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [47268.576352] Lustre: Skipped 3 previous similar messages [47305.171029] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [47305.184357] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [47313.795853] Lustre: soaked-OST0011: deleting orphan objects from 0x0:15058713 to 0x0:15072145 [47313.795855] Lustre: soaked-OST0017: deleting orphan objects from 0x0:15103964 to 0x0:15113489 [47313.795858] Lustre: soaked-OST0005: deleting orphan objects from 0x0:15166430 to 0x0:15176353 [47313.795862] Lustre: soaked-OST000b: deleting orphan objects from 0x0:15105417 to 0x0:15119233 [47337.743943] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [47350.522318] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.109@o2ib) was lost; in progress operations using this service will fail [47375.543422] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xf8a25437a9cd4c5b to 0x78d1c7ef35bcc77 [47375.563941] Lustre: MGC192.168.1.108@o2ib: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [47375.578308] Lustre: Skipped 3 previous similar messages [47383.679117] Lustre: soaked-OST0005: deleting orphan objects from 0x0:15180565 to 0x0:15181217 [47383.679119] Lustre: soaked-OST000b: deleting orphan objects from 0x0:15123180 to 0x0:15128353 [47383.679138] Lustre: soaked-OST0011: deleting orphan objects from 0x0:15075850 to 0x0:15080753 [47383.679142] Lustre: soaked-OST0017: deleting orphan objects from 0x0:15117267 to 0x0:15122321 [47416.518217] Lustre: soaked-MDT0000-lwp-OST0011: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [47416.518217] Lustre: soaked-MDT0000-lwp-OST0017: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [47416.560534] Lustre: Skipped 2 previous similar messages [47441.565982] LustreError: 167-0: soaked-MDT0000-lwp-OST0005: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [47441.585149] Lustre: soaked-MDT0000-lwp-OST0005: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [47441.598021] Lustre: Skipped 3 previous similar messages [49423.449889] Lustre: 21474:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510847098/real 0] req@ffff88029b5d4500 x1584189547564208/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510847105 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [49423.790006] Lustre: 21474:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 10 previous similar messages [49423.910766] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [49440.541029] Lustre: soaked-MDT0000-lwp-OST0017: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [49440.743663] Lustre: Skipped 3 previous similar messages [49465.806763] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510847147/real 1510847147] req@ffff88029b5d0000 x1584189547564448/t0(0) o38->soaked-MDT0000-lwp-OST0017@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510847158 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [49466.164877] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 10 previous similar messages [49556.800765] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510847222/real 1510847222] req@ffff88029b5d6f00 x1584189547564688/t0(0) o38->soaked-MDT0000-lwp-OST0017@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510847238 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [49556.842156] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 18 previous similar messages [49639.917127] Lustre: soaked-OST0005: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff88029f235000, cur 1510847321 expire 1510847171 last 1510847094 [49641.726811] Lustre: soaked-OST000b: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff88024dc0c400, cur 1510847323 expire 1510847173 last 1510847096 [49654.027846] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 4 seconds [49691.025607] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 0 seconds [49721.790695] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510847372/real 0] req@ffff88029b5d0600 x1584189547565024/t0(0) o38->soaked-MDT0000-lwp-OST0017@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510847403 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [49721.892625] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 20 previous similar messages [49741.022555] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 0 seconds [49741.059765] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [49838.635632] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [49838.729823] Lustre: Skipped 3 previous similar messages [49840.570416] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0017: can't get bonus, rc = -17 [49840.701138] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 2 previous similar messages [49848.280918] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0017: can't get bonus, rc = -17 [49848.411560] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 154 previous similar messages [49865.782595] LustreError: 167-0: soaked-MDT0000-lwp-OST000b: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [49865.952416] LustreError: Skipped 3 previous similar messages [49866.023494] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0x78d1c7ef35bcc77 to 0x485b513ca2b51663 [49866.024196] Lustre: soaked-MDT0000-lwp-OST000b: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [49866.299965] LustreError: 167-0: soaked-MDT0000-lwp-OST0005: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [49866.472183] LustreError: Skipped 1 previous similar message [49915.779443] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [49915.949648] Lustre: soaked-MDT0000-lwp-OST0017: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [49916.074003] Lustre: Skipped 3 previous similar messages [49919.011659] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 20 seconds [49965.226790] Lustre: soaked-OST0005: deleting orphan objects from 0x0:17018162 to 0x0:17024977 [49965.226830] Lustre: soaked-OST000b: deleting orphan objects from 0x0:16965598 to 0x0:16976449 [49965.226831] Lustre: soaked-OST0011: deleting orphan objects from 0x0:16919177 to 0x0:16925393 [49965.226834] Lustre: soaked-OST0017: deleting orphan objects from 0x0:16959633 to 0x0:16968481 [50115.616585] LustreError: 188018:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0005: can't get bonus, rc = -17 [50115.653606] LustreError: 188018:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 35 previous similar messages [50130.039699] LustreError: 188018:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0005: can't get bonus, rc = -17 [50130.076964] LustreError: 188018:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 185 previous similar messages [50163.737665] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0017: can't get bonus, rc = -17 [50163.774486] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 229 previous similar messages [50198.978501] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0017: can't get bonus, rc = -17 [50199.015258] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 139 previous similar messages [50285.613212] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) soaked-OST0017: can't get bonus, rc = -17 [50285.743602] LustreError: 188021:0:(osd_index.c:223:__osd_xattr_load_by_oid()) Skipped 362 previous similar messages [51041.943164] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [51041.979778] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [51647.906215] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 2 seconds [51950.887662] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [53072.248930] Lustre: 21476:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510850747/real 1510850754] req@ffff8803faeb5d00 x1584189603954624/t0(0) o400->soaked-MDT0000-lwp-OST000b@192.168.1.108@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510850819 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [53072.248956] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [53072.248962] Lustre: soaked-MDT0000-lwp-OST0011: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [53072.987652] Lustre: 21476:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 23 previous similar messages [53073.108178] Lustre: soaked-MDT0000-lwp-OST000b: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [53073.313765] Lustre: Skipped 2 previous similar messages [53123.376584] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510850805/real 1510850805] req@ffff88029b5d3000 x1584189603954864/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510850816 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [53123.733760] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 6 previous similar messages [53213.810701] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 4 seconds [53214.370662] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510850880/real 1510850880] req@ffff88029b5d6900 x1584189603955120/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510850896 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [53214.411660] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 14 previous similar messages [53270.340260] Lustre: soaked-OST0017: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8807657e6000, cur 1510850952 expire 1510850802 last 1510850725 [53270.370028] Lustre: Skipped 2 previous similar messages [53271.212693] Lustre: soaked-OST0005: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8807b3ec4400, cur 1510850953 expire 1510850803 last 1510850726 [53271.242525] Lustre: Skipped 1 previous similar message [53272.595073] Lustre: soaked-OST0011: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8807f99bdc00, cur 1510850954 expire 1510850804 last 1510850727 [53276.806929] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 2 seconds [53314.642274] Lustre: soaked-OST0005: Connection restored to soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) [53314.656479] Lustre: Skipped 1 previous similar message [53326.803820] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 4 seconds [53339.803015] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 5 seconds [53339.817634] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [53348.363113] LustreError: 167-0: soaked-MDT0000-lwp-OST000b: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [53348.385351] Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib_1) after server handle changed from 0x485b513ca2b51663 to 0xf8a25437aa14875b [53348.386192] Lustre: soaked-MDT0000-lwp-OST000b: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [53348.386195] Lustre: Skipped 2 previous similar messages [53412.623709] Lustre: soaked-OST000b: deleting orphan objects from 0x0:18388817 to 0x0:18407345 [53412.623712] Lustre: soaked-OST0005: deleting orphan objects from 0x0:18438446 to 0x0:18443905 [53412.623719] Lustre: soaked-OST0011: deleting orphan objects from 0x0:18339954 to 0x0:18352161 [53412.623724] Lustre: soaked-OST0017: deleting orphan objects from 0x0:18378749 to 0x0:18388865 [53423.358083] LustreError: 11-0: soaked-MDT0000-lwp-OST0011: operation obd_ping to node 192.168.1.109@o2ib failed: rc = -107 [53423.358099] Lustre: soaked-MDT0000-lwp-OST0005: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [53423.358101] Lustre: Skipped 1 previous similar message [53423.473483] LustreError: Skipped 3 previous similar messages [53440.902984] Lustre: soaked-OST000b: Connection restored to (at 192.168.1.108@o2ib) [53440.929785] Lustre: Skipped 6 previous similar messages [53448.356922] LustreError: 167-0: soaked-MDT0000-lwp-OST0005: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [53448.406285] LustreError: Skipped 3 previous similar messages [53455.356035] Lustre: 21477:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510851130/real 1510851130] req@ffff88029b5d7800 x1584189605144704/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.109@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510851137 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [53455.459247] Lustre: 21477:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 19 previous similar messages [53455.494858] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.109@o2ib) was lost; in progress operations using this service will fail [53480.547026] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xf8a25437aa14875b to 0x70992b92de64279b [53480.593669] Lustre: MGC192.168.1.108@o2ib: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [53480.628350] Lustre: Skipped 4 previous similar messages [53519.109511] Lustre: soaked-OST000b: deleting orphan objects from 0x0:18411314 to 0x0:18411441 [53519.109513] Lustre: soaked-OST0011: deleting orphan objects from 0x0:18356425 to 0x0:18356609 [53519.109516] Lustre: soaked-OST0017: deleting orphan objects from 0x0:18393145 to 0x0:18394657 [53519.109518] Lustre: soaked-OST0005: deleting orphan objects from 0x0:18447298 to 0x0:18447809 [57662.299393] Lustre: 21478:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510855337/real 0] req@ffff8803f4765d00 x1584189691215072/t0(0) o400->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 224/224 e 0 to 1 dl 1510855344 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [57662.398770] Lustre: 21478:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 1 previous similar message [57662.433562] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [57680.221771] Lustre: soaked-MDT0000-lwp-OST0011: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [57680.281021] Lustre: Skipped 5 previous similar messages [57705.299454] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510855387/real 1510855387] req@ffff8803f4763f00 x1584189691215296/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510855398 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [57705.403662] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 10 previous similar messages [57733.535084] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [57796.293231] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510855462/real 1510855462] req@ffff8803f4765a00 x1584189691215536/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510855478 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [57796.531177] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [57796.772220] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 18 previous similar messages [57858.120985] Lustre: soaked-OST0017: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 228 seconds. I think it's dead, and I am evicting it. exp ffff880327f7d000, cur 1510855540 expire 1510855390 last 1510855312 [57858.833384] Lustre: soaked-OST0011: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8803f0273c00, cur 1510855541 expire 1510855391 last 1510855314 [57862.976336] Lustre: soaked-OST000b: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 229 seconds. I think it's dead, and I am evicting it. exp ffff88042a0e7400, cur 1510855545 expire 1510855395 last 1510855316 [57863.006937] Lustre: Skipped 1 previous similar message [57899.013782] Lustre: soaked-OST0005: Connection restored to soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) [57899.028321] Lustre: Skipped 1 previous similar message [57909.524454] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 5 seconds [57930.285756] LustreError: 167-0: soaked-MDT0000-lwp-OST0011: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [57930.305377] LustreError: Skipped 3 previous similar messages [57930.316479] Lustre: soaked-MDT0000-lwp-OST0011: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [57930.331508] Lustre: Skipped 2 previous similar messages [57934.522824] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 5 seconds [57955.284215] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [57955.303842] Lustre: soaked-MDT0000-lwp-OST0005: Connection restored to 192.168.1.109@o2ib (at 192.168.1.109@o2ib) [57955.316854] Lustre: Skipped 2 previous similar messages [57955.333722] Lustre: Evicted from MGS (at MGC192.168.1.108@o2ib_1) after server handle changed from 0x70992b92de64279b to 0xf8a25437aa4a6792 [58036.516552] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [58036.553797] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [58465.490387] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 4 seconds [58705.475767] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [59563.423425] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 2 seconds [59563.438316] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [59866.404953] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [59866.442129] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [60232.382628] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 2 seconds [60351.815856] Lustre: soaked-OST0017: haven't heard from client b1010b82-5169-87e5-3f1a-977d836fd667 (at 192.168.1.143@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8800ac40e800, cur 1510858034 expire 1510857884 last 1510857807 [61394.311784] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [61394.437951] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [61487.068148] Lustre: 21481:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510859162/real 0] req@ffff880259334e00 x1584189691353776/t0(0) o400->soaked-MDT0000-lwp-OST000b@192.168.1.109@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510859169 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [61487.068152] Lustre: 21483:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510859162/real 0] req@ffff880259331b00 x1584189691353808/t0(0) o400->soaked-MDT0000-lwp-OST0017@192.168.1.109@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510859169 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [61487.068156] Lustre: 21482:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510859162/real 0] req@ffff880259332400 x1584189691353792/t0(0) o400->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510859169 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [61487.068160] Lustre: 21480:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510859162/real 0] req@ffff880259331800 x1584189691353760/t0(0) o400->soaked-MDT0000-lwp-OST0005@192.168.1.109@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510859169 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [61487.068163] Lustre: 21483:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 19 previous similar messages [61487.068165] Lustre: 21482:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 20 previous similar messages [61487.068171] Lustre: 21480:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 20 previous similar messages [61487.068177] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.109@o2ib) was lost; in progress operations using this service will fail [61487.068179] Lustre: soaked-MDT0000-lwp-OST0017: Connection to soaked-MDT0000 (at 192.168.1.109@o2ib) was lost; in progress operations using this service will wait for recovery to complete [61493.318737] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510859169/real 0] req@ffff880259335700 x1584189691353888/t0(0) o38->soaked-MDT0000-lwp-OST000b@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510859175 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [61512.317042] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510859194/real 1510859194] req@ffff880259330f00 x1584189691353952/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510859200 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [61512.359362] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [61537.315584] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510859219/real 1510859219] req@ffff880259334500 x1584189691354032/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510859230 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [61537.358069] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [61562.314042] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510859244/real 1510859244] req@ffff880259337b00 x1584189691354112/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510859255 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [61562.355678] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 4 previous similar messages [61570.918971] Lustre: soaked-OST0005: haven't heard from client 1670ceaa-82d1-039a-867b-121b2da9a089 (at 192.168.1.142@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8803f6065c00, cur 1510859253 expire 1510859103 last 1510859026 [61570.950347] Lustre: Skipped 3 previous similar messages [61571.565789] Lustre: soaked-OST0017: haven't heard from client 1670ceaa-82d1-039a-867b-121b2da9a089 (at 192.168.1.142@o2ib) in 228 seconds. I think it's dead, and I am evicting it. exp ffff88040fff1400, cur 1510859254 expire 1510859104 last 1510859026 [61573.111470] Lustre: soaked-OST000b: haven't heard from client 1670ceaa-82d1-039a-867b-121b2da9a089 (at 192.168.1.142@o2ib) in 229 seconds. I think it's dead, and I am evicting it. exp ffff8803f7e7ac00, cur 1510859255 expire 1510859105 last 1510859026 [61612.310869] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510859294/real 1510859294] req@ffff880259331e00 x1584189691354272/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510859310 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [61612.354444] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 9 previous similar messages [61687.306320] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510859369/real 1510859369] req@ffff880259337200 x1584189691354512/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510859395 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [61687.413083] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 14 previous similar messages [61697.293284] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 4 seconds [61697.330728] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [61703.177970] Lustre: soaked-OST0011: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.109@o2ib) in 229 seconds. I think it's dead, and I am evicting it. exp ffff8802c7a50800, cur 1510859385 expire 1510859235 last 1510859156 [61703.255292] Lustre: Skipped 1 previous similar message [61772.288731] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.108@o2ib: 10 seconds [61840.578510] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [61840.590644] Lustre: Skipped 2 previous similar messages [61848.284110] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.109@o2ib: 11 seconds [61848.299229] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [61862.340246] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0xf8a25437aa4a6792 to 0x18357c4533462bc8 [61862.361056] Lustre: MGC192.168.1.108@o2ib: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [61862.375685] Lustre: Skipped 2 previous similar messages [61912.292683] LustreError: 167-0: soaked-MDT0000-lwp-OST0005: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [61912.315476] LustreError: Skipped 2 previous similar messages [61912.326726] Lustre: soaked-MDT0000-lwp-OST0005: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [61912.341898] Lustre: Skipped 3 previous similar messages [62063.270969] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [62063.397396] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [62492.244790] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [62492.371086] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [62795.226328] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 1 seconds [63869.172823] Lustre: 21474:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1510861544/real 1510861544] req@ffff880259334800 x1584189691361792/t0(0) o400->soaked-MDT0000-lwp-OST0017@192.168.1.108@o2ib:12/10 lens 224/224 e 0 to 1 dl 1510861551 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1 [63869.172836] LustreError: 166-1: MGC192.168.1.108@o2ib: Connection to MGS (at 192.168.1.108@o2ib) was lost; in progress operations using this service will fail [63869.172839] Lustre: soaked-MDT0000-lwp-OST000b: Connection to soaked-MDT0000 (at 192.168.1.108@o2ib) was lost; in progress operations using this service will wait for recovery to complete [63869.172841] Lustre: Skipped 3 previous similar messages [63869.270623] Lustre: 21474:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 18 previous similar messages [63894.159367] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [63894.174212] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [63900.281976] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510861576/real 0] req@ffff880259332400 x1584189691361936/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.109@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510861582 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [63900.323196] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 5 previous similar messages [63944.279727] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510861626/real 1510861626] req@ffff880259334e00 x1584189691362032/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510861637 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [63944.322054] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 5 previous similar messages [64019.276311] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1510861701/real 1510861701] req@ffff880259332400 x1584189691362192/t0(0) o38->soaked-MDT0000-lwp-OST0011@192.168.1.108@o2ib:12/10 lens 520/544 e 0 to 1 dl 1510861717 ref 1 fl Rpc:eXN/0/ffffffff rc 0/-1 [64019.382167] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 9 previous similar messages [64068.357139] Lustre: soaked-OST0011: haven't heard from client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8807d797e000, cur 1510861750 expire 1510861600 last 1510861523 [64068.433940] Lustre: Skipped 3 previous similar messages [64170.265479] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1510861826/real 0] req@ffff880259335100 x1584189691362384/t0(0) o250->MGC192.168.1.108@o2ib@192.168.1.108@o2ib:26/25 lens 520/544 e 0 to 1 dl 1510861852 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [64170.606388] Lustre: 21471:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 14 previous similar messages [64274.049855] Lustre: soaked-OST0005: Connection restored to (at 192.168.1.108@o2ib) [64294.303151] Lustre: Evicted from MGS (at 192.168.1.108@o2ib) after server handle changed from 0x18357c4533462bc8 to 0xbab2a087086d90ac [64294.323736] Lustre: MGC192.168.1.108@o2ib: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [64294.338169] Lustre: Skipped 3 previous similar messages [64394.252426] LustreError: 167-0: soaked-MDT0000-lwp-OST0017: This client was evicted by soaked-MDT0000; in progress operations using this service will fail. [64394.305041] LustreError: Skipped 3 previous similar messages [64394.328520] Lustre: soaked-MDT0000-lwp-OST0017: Connection restored to 192.168.1.108@o2ib (at 192.168.1.108@o2ib) [64394.365771] Lustre: Skipped 3 previous similar messages [64626.114650] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [64626.151840] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 30 previous similar messages [65295.073870] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [66027.029210] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 1 seconds [66027.066318] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [66822.980667] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 3 seconds [66823.017776] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [67554.936025] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 4 seconds [67554.950775] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [68526.876759] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 1 seconds [68526.891424] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [69168.541676] Lustre: soaked-OST0005: deleting orphan objects from 0x0:20505469 to 0x0:20517649 [69168.541683] Lustre: soaked-OST000b: deleting orphan objects from 0x0:20453104 to 0x0:20463329 [69168.541692] Lustre: soaked-OST0017: deleting orphan objects from 0x0:20447051 to 0x0:20457393 [69168.541696] Lustre: soaked-OST0011: deleting orphan objects from 0x0:20407577 to 0x0:20418433 [69170.444627] Lustre: soaked-OST000b: Connection restored to 1670ceaa-82d1-039a-867b-121b2da9a089 (at 192.168.1.142@o2ib) [69170.481831] Lustre: Skipped 3 previous similar messages [69283.081387] Lustre: soaked-OST000b: Connection restored to b1010b82-5169-87e5-3f1a-977d836fd667 (at 192.168.1.143@o2ib) [69283.213583] Lustre: Skipped 7 previous similar messages [69321.828264] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 1 seconds [69321.954194] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 4 previous similar messages [70353.031132] Lustre: soaked-OST0017: haven't heard from client 43d65688-cf88-a063-15b1-00edcb601bee (at 192.168.1.141@o2ib) in 228 seconds. I think it's dead, and I am evicting it. exp ffff8803eefff000, cur 1510868036 expire 1510867886 last 1510867808 [70353.061835] Lustre: Skipped 3 previous similar messages [70470.578845] Lustre: soaked-OST0011: haven't heard from client 9972e6fb-d5a8-54ee-8435-a4d933d3b728 (at 192.168.1.140@o2ib) in 228 seconds. I think it's dead, and I am evicting it. exp ffff88080f99c400, cur 1510868153 expire 1510868003 last 1510867925 [70470.657371] Lustre: Skipped 3 previous similar messages [70786.738914] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [71152.716596] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [71455.698130] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [72250.649656] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [72250.663943] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [72983.604928] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [73437.834533] Lustre: soaked-OST000b: Connection restored to 9972e6fb-d5a8-54ee-8435-a4d933d3b728 (at 192.168.1.140@o2ib) [73437.849389] Lustre: Skipped 3 previous similar messages [73563.675710] Lustre: soaked-OST0005: Connection restored to 43d65688-cf88-a063-15b1-00edcb601bee (at 192.168.1.141@o2ib) [73563.807522] Lustre: Skipped 3 previous similar messages [73652.564125] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 1 seconds [73652.578302] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [77326.340066] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [77326.465420] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message [77489.671101] LustreError: 0:0:(ldlm_lockd.c:331:waiting_locks_callback()) ### lock callback timer expired after 100s: evicting client at 192.168.1.124@o2ib ns: filter-soaked-OST0017_UUID lock: ffff8805bc498a00/0x36a50a0a07868f4b lrc: 3/0,0 mode: PW/PW res: [0x153c617:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) flags: 0x60000400000020 nid: 192.168.1.124@o2ib remote: 0x1638c49dd51dc4cf expref: 6 pid: 62809 timeout: 77494 lvb_type: 0 [77490.595774] Lustre: soaked-OST0017: Connection restored to 6613643f-e3f6-1044-83bc-aef6b27c6309 (at 192.168.1.124@o2ib) [77589.665002] LustreError: 0:0:(ldlm_lockd.c:331:waiting_locks_callback()) ### lock callback timer expired after 100s: evicting client at 192.168.1.140@o2ib ns: filter-soaked-OST0017_UUID lock: ffff8806fa137e00/0x36a50a0a0797452b lrc: 3/0,0 mode: PW/PW res: [0x153c617:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) flags: 0x60000000020020 nid: 192.168.1.140@o2ib remote: 0xed32af8c7ff3b01f expref: 5 pid: 116364 timeout: 77594 lvb_type: 0 [77615.235582] Lustre: soaked-OST0017: Connection restored to 9972e6fb-d5a8-54ee-8435-a4d933d3b728 (at 192.168.1.140@o2ib) [77689.658915] LustreError: 0:0:(ldlm_lockd.c:331:waiting_locks_callback()) ### lock callback timer expired after 100s: evicting client at 192.168.1.141@o2ib ns: filter-soaked-OST0017_UUID lock: ffff880239601e00/0x36a50a0a07974532 lrc: 3/0,0 mode: PW/PW res: [0x153c617:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) flags: 0x60000000020020 nid: 192.168.1.141@o2ib remote: 0xa129d0bfbf59cf9f expref: 9 pid: 38246 timeout: 77694 lvb_type: 0 [77692.317751] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 0 seconds [77708.255933] Lustre: soaked-OST0017: Connection restored to 43d65688-cf88-a063-15b1-00edcb601bee (at 192.168.1.141@o2ib) [77718.378719] Lustre: soaked-OST0017: haven't heard from client 6613643f-e3f6-1044-83bc-aef6b27c6309 (at 192.168.1.124@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8807ffa62400, cur 1510875401 expire 1510875251 last 1510875174 [77718.407975] Lustre: Skipped 3 previous similar messages [77789.652795] LustreError: 0:0:(ldlm_lockd.c:331:waiting_locks_callback()) ### lock callback timer expired after 100s: evicting client at 192.168.1.128@o2ib ns: filter-soaked-OST0017_UUID lock: ffff8802f4bf1400/0x36a50a0a07974547 lrc: 3/0,0 mode: PW/PW res: [0x153c617:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) flags: 0x60000000020020 nid: 192.168.1.128@o2ib remote: 0xbb7b608ec20b51e0 expref: 6 pid: 172245 timeout: 77794 lvb_type: 0 [77797.954882] Lustre: soaked-OST0017: Connection restored to 8b0c50cd-9f3a-6240-b7ca-f83ce30a2cb0 (at 192.168.1.128@o2ib) [77841.611058] Lustre: soaked-OST0017: haven't heard from client 8acfcce2-0a6a-7a64-c343-af6ae3441746 (at 192.168.1.140@o2ib) in 227 seconds. I think it's dead, and I am evicting it. exp ffff8801d00b1000, cur 1510875525 expire 1510875375 last 1510875298 [77889.646708] LustreError: 0:0:(ldlm_lockd.c:331:waiting_locks_callback()) ### lock callback timer expired after 100s: evicting client at 192.168.1.142@o2ib ns: filter-soaked-OST0017_UUID lock: ffff8805cbadd600/0x36a50a0a0797455c lrc: 3/0,0 mode: PW/PW res: [0x153c617:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) flags: 0x60000000020020 nid: 192.168.1.142@o2ib remote: 0xb30c49e921aff9c5 expref: 6 pid: 180477 timeout: 77894 lvb_type: 0 [77890.355084] Lustre: soaked-OST0017: Connection restored to 1670ceaa-82d1-039a-867b-121b2da9a089 (at 192.168.1.142@o2ib) [77917.542906] Lustre: soaked-OST0017: haven't heard from client 00eb46e8-271c-6604-bab8-c7a5b945c161 (at 192.168.1.141@o2ib) in 210 seconds. I think it's dead, and I am evicting it. exp ffff8803821ad000, cur 1510875601 expire 1510875451 last 1510875391 [77989.640638] LustreError: 0:0:(ldlm_lockd.c:331:waiting_locks_callback()) ### lock callback timer expired after 100s: evicting client at 192.168.1.143@o2ib ns: filter-soaked-OST0017_UUID lock: ffff8804f5b40600/0x36a50a0a07974571 lrc: 3/0,0 mode: PW/PW res: [0x153c617:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) flags: 0x60000000020020 nid: 192.168.1.143@o2ib remote: 0x7acfa5e11eb74cd2 expref: 5 pid: 116421 timeout: 77994 lvb_type: 0 [77990.176840] LustreError: 12092:0:(ldlm_lockd.c:2334:ldlm_cancel_handler()) ldlm_cancel from 192.168.1.143@o2ib arrived at 1510875673 with bad export cookie 3937564487323952175 [77990.432882] Lustre: soaked-OST0017: Connection restored to b1010b82-5169-87e5-3f1a-977d836fd667 (at 192.168.1.143@o2ib) [77995.299215] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [78116.161091] INFO: task txg_quiesce:13427 blocked for more than 120 seconds. [78116.172661] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78116.184879] txg_quiesce D 0000000000000000 0 13427 2 0x00000080 [78116.195983] ffff8803f621bd38 0000000000000046 ffff8803f6180fd0 ffff8803f621bfd8 [78116.207348] ffff8803f621bfd8 ffff8803f621bfd8 ffff8803f6180fd0 ffff8803f6082290 [78116.218533] ffff8803f6082180 ffff8803f60821a8 ffff8803f6082298 0000000000000000 [78116.229495] Call Trace: [78116.234646] [] schedule+0x29/0x70 [78116.242643] [] cv_wait_common+0x125/0x150 [spl] [78116.251809] [] ? wake_up_atomic_t+0x30/0x30 [78116.260592] [] __cv_wait+0x15/0x20 [spl] [78116.268914] [] txg_quiesce_thread+0x2fb/0x410 [zfs] [78116.278333] [] ? txg_init+0x2b0/0x2b0 [zfs] [78116.286759] [] thread_generic_wrapper+0x71/0x80 [spl] [78116.296152] [] ? __thread_exit+0x20/0x20 [spl] [78116.304737] [] kthread+0xcf/0xe0 [78116.311954] [] ? insert_kthread_work+0x40/0x40 [78116.320433] [] ret_from_fork+0x58/0x90 [78116.328079] [] ? insert_kthread_work+0x40/0x40 [78116.336464] INFO: task txg_quiesce:13663 blocked for more than 120 seconds. [78116.347098] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78116.358878] txg_quiesce D 0000000000000000 0 13663 2 0x00000080 [78116.369585] ffff8803f5a9fd38 0000000000000046 ffff8803f59e0fd0 ffff8803f5a9ffd8 [78116.380518] ffff8803f5a9ffd8 ffff8803f5a9ffd8 ffff8803f59e0fd0 ffff8803f5901be8 [78116.391196] ffff8803f5901ad8 ffff8803f5901b00 ffff8803f5901bf0 0000000000000000 [78116.401715] Call Trace: [78116.406398] [] schedule+0x29/0x70 [78116.413911] [] cv_wait_common+0x125/0x150 [spl] [78116.422596] [] ? wake_up_atomic_t+0x30/0x30 [78116.430864] [] __cv_wait+0x15/0x20 [spl] [78116.438761] [] txg_quiesce_thread+0x2fb/0x410 [zfs] [78116.447695] [] ? txg_init+0x2b0/0x2b0 [zfs] [78116.455714] [] thread_generic_wrapper+0x71/0x80 [spl] [78116.464707] [] ? __thread_exit+0x20/0x20 [spl] [78116.472879] [] kthread+0xcf/0xe0 [78116.479694] [] ? insert_kthread_work+0x40/0x40 [78116.487748] [] ret_from_fork+0x58/0x90 [78116.495038] [] ? insert_kthread_work+0x40/0x40 [78116.502999] INFO: task ll_ost_out01_00:21974 blocked for more than 120 seconds. [78116.512326] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78116.522136] ll_ost_out01_00 D ffffffff00000000 0 21974 2 0x00000080 [78116.531063] ffff8803f3b3fa28 0000000000000046 ffff88042c968000 ffff8803f3b3ffd8 [78116.540384] ffff8803f3b3ffd8 ffff8803f3b3ffd8 ffff88042c968000 ffff88042c968000 [78116.549634] ffff8806acc9e020 ffff8806acc9e018 ffffffff00000008 ffffffff00000000 [78116.558842] Call Trace: [78116.562421] [] schedule+0x29/0x70 [78116.568802] [] rwsem_down_write_failed+0x225/0x3a0 [78116.576810] [] call_rwsem_down_write_failed+0x17/0x30 [78116.585093] [] ? spl_kmem_alloc+0xc7/0x170 [spl] [78116.592877] [] down_write+0x2d/0x3d [78116.599408] [] osd_write_lock+0x39/0x40 [osd_zfs] [78116.607348] [] out_tx_attr_set_exec+0x66/0x410 [ptlrpc] [78116.615841] [] out_tx_end+0xde/0x5c0 [ptlrpc] [78116.623382] [] out_handle+0x11e7/0x1920 [ptlrpc] [78116.631155] [] ? update_curr+0x104/0x190 [78116.638179] [] ? lustre_msg_get_opc+0x22/0xf0 [ptlrpc] [78116.646572] [] ? tgt_request_preprocess.isra.26+0x299/0x7a0 [ptlrpc] [78116.656352] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78116.664836] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78116.674211] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78116.682583] [] ? default_wake_function+0x12/0x20 [78116.690365] [] ? __wake_up_common+0x58/0x90 [78116.697704] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78116.705513] [] ? __switch_to+0xd7/0x510 [78116.712430] [] ? __schedule+0x350/0x8b0 [78116.719379] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78116.728437] [] kthread+0xcf/0xe0 [78116.734666] [] ? insert_kthread_work+0x40/0x40 [78116.742250] [] ret_from_fork+0x58/0x90 [78116.749044] [] ? insert_kthread_work+0x40/0x40 [78116.756612] INFO: task ll_ost_io01_004:46346 blocked for more than 120 seconds. [78116.765555] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78116.775087] ll_ost_io01_004 D 0000000000000000 0 46346 2 0x00000080 [78116.783795] ffff8803f021b830 0000000000000046 ffff8808051b9fa0 ffff8803f021bfd8 [78116.792932] ffff8803f021bfd8 ffff8803f021bfd8 ffff8808051b9fa0 ffff88017912a388 [78116.802041] ffff88017912a240 ffff88017912a268 ffff88017912a390 0000000000000000 [78116.811167] Call Trace: [78116.814687] [] schedule+0x29/0x70 [78116.821027] [] cv_wait_common+0x125/0x150 [spl] [78116.828729] [] ? wake_up_atomic_t+0x30/0x30 [78116.836041] [] __cv_wait+0x15/0x20 [spl] [78116.843082] [] txg_wait_open+0xc3/0x110 [zfs] [78116.850609] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78116.858033] [] dmu_tx_assign+0x91/0x490 [zfs] [78116.865541] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78116.873639] [] ofd_trans_start+0x6b/0xe0 [ofd] [78116.881243] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78116.889416] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78116.896904] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78116.904692] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78116.912646] [] ? update_curr+0x104/0x190 [78116.919618] [] ? __enqueue_entity+0x78/0x80 [78116.926858] [] ? enqueue_entity+0x26c/0xb60 [78116.934130] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [78116.942968] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78116.951433] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78116.960765] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78116.969089] [] ? default_wake_function+0x12/0x20 [78116.976864] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78116.984641] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78116.993661] [] kthread+0xcf/0xe0 [78116.999848] [] ? insert_kthread_work+0x40/0x40 [78117.007411] [] ret_from_fork+0x58/0x90 [78117.014185] [] ? insert_kthread_work+0x40/0x40 [78117.021736] INFO: task ll_ost_io01_010:52493 blocked for more than 120 seconds. [78117.030651] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78117.040154] ll_ost_io01_010 D 0000000000000000 0 52493 2 0x00000080 [78117.048836] ffff8807b73eb830 0000000000000046 ffff880800758fd0 ffff8807b73ebfd8 [78117.057945] ffff8807b73ebfd8 ffff8807b73ebfd8 ffff880800758fd0 ffff88017912a388 [78117.067057] ffff88017912a240 ffff88017912a268 ffff88017912a390 0000000000000000 [78117.076146] Call Trace: [78117.079632] [] schedule+0x29/0x70 [78117.085944] [] cv_wait_common+0x125/0x150 [spl] [78117.093619] [] ? wake_up_atomic_t+0x30/0x30 [78117.100866] [] __cv_wait+0x15/0x20 [spl] [78117.107839] [] txg_wait_open+0xc3/0x110 [zfs] [78117.115302] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78117.122631] [] dmu_tx_assign+0x91/0x490 [zfs] [78117.130060] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78117.138021] [] ofd_trans_start+0x6b/0xe0 [ofd] [78117.145474] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78117.153475] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78117.160850] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78117.168480] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78117.176271] [] ? update_curr+0x104/0x190 [78117.183092] [] ? __enqueue_entity+0x78/0x80 [78117.190205] [] ? enqueue_entity+0x26c/0xb60 [78117.197355] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [78117.206007] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78117.214291] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78117.223433] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78117.231579] [] ? default_wake_function+0x12/0x20 [78117.239150] [] ? __wake_up_common+0x58/0x90 [78117.246282] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78117.253886] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78117.262719] [] kthread+0xcf/0xe0 [78117.268751] [] ? insert_kthread_work+0x40/0x40 [78117.276165] [] ret_from_fork+0x58/0x90 [78117.282785] [] ? insert_kthread_work+0x40/0x40 [78117.290182] INFO: task ll_ost_io01_016:52549 blocked for more than 120 seconds. [78117.298965] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78117.308310] ll_ost_io01_016 D 0000000000000000 0 52549 2 0x00000080 [78117.316830] ffff8807bc02b830 0000000000000046 ffff8803efe08000 ffff8807bc02bfd8 [78117.325745] ffff8807bc02bfd8 ffff8807bc02bfd8 ffff8803efe08000 ffff880823718b88 [78117.334657] ffff880823718a40 ffff880823718a68 ffff880823718b90 0000000000000000 [78117.343586] Call Trace: [78117.346919] [] schedule+0x29/0x70 [78117.353069] [] cv_wait_common+0x125/0x150 [spl] [78117.360570] [] ? wake_up_atomic_t+0x30/0x30 [78117.367681] [] __cv_wait+0x15/0x20 [spl] [78117.374522] [] txg_wait_open+0xc3/0x110 [zfs] [78117.381872] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78117.389094] [] dmu_tx_assign+0x91/0x490 [zfs] [78117.396392] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78117.404262] [] ofd_trans_start+0x6b/0xe0 [ofd] [78117.411640] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78117.419603] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78117.426933] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78117.434521] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78117.442293] [] ? update_curr+0x104/0x190 [78117.449066] [] ? __enqueue_entity+0x78/0x80 [78117.456173] [] ? enqueue_entity+0x26c/0xb60 [78117.463291] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [78117.471968] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78117.480261] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78117.489398] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78117.497540] [] ? default_wake_function+0x12/0x20 [78117.505130] [] ? __wake_up_common+0x58/0x90 [78117.512236] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78117.519853] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78117.528694] [] kthread+0xcf/0xe0 [78117.534728] [] ? insert_kthread_work+0x40/0x40 [78117.542116] [] ret_from_fork+0x58/0x90 [78117.548741] [] ? insert_kthread_work+0x40/0x40 [78117.556151] INFO: task ll_ost_io01_017:52550 blocked for more than 120 seconds. [78117.564932] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78117.574300] ll_ost_io01_017 D 0000000000000000 0 52550 2 0x00000080 [78117.582833] ffff8807c0a8b830 0000000000000046 ffff8803efe0eeb0 ffff8807c0a8bfd8 [78117.591762] ffff8807c0a8bfd8 ffff8807c0a8bfd8 ffff8803efe0eeb0 ffff880823718b88 [78117.600664] ffff880823718a40 ffff880823718a68 ffff880823718b90 0000000000000000 [78117.609576] Call Trace: [78117.612926] [] schedule+0x29/0x70 [78117.619100] [] cv_wait_common+0x125/0x150 [spl] [78117.626598] [] ? wake_up_atomic_t+0x30/0x30 [78117.633720] [] __cv_wait+0x15/0x20 [spl] [78117.640566] [] txg_wait_open+0xc3/0x110 [zfs] [78117.647918] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78117.655166] [] dmu_tx_assign+0x91/0x490 [zfs] [78117.662460] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78117.670341] [] ofd_trans_start+0x6b/0xe0 [ofd] [78117.677731] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78117.685689] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78117.693016] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78117.700600] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78117.708352] [] ? update_curr+0x104/0x190 [78117.715163] [] ? __enqueue_entity+0x78/0x80 [78117.722231] [] ? enqueue_entity+0x26c/0xb60 [78117.729325] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [78117.738002] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78117.746267] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78117.755412] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78117.763562] [] ? default_wake_function+0x12/0x20 [78117.771151] [] ? __wake_up_common+0x58/0x90 [78117.778291] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78117.785920] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78117.794789] [] kthread+0xcf/0xe0 [78117.800846] [] ? insert_kthread_work+0x40/0x40 [78117.808259] [] ret_from_fork+0x58/0x90 [78117.814873] [] ? insert_kthread_work+0x40/0x40 [78117.822291] INFO: task ll_ost_io01_021:52731 blocked for more than 120 seconds. [78117.831072] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78117.840418] ll_ost_io01_021 D 0000000000000000 0 52731 2 0x00000080 [78117.848960] ffff8803c81e3830 0000000000000046 ffff8803eecc5ee0 ffff8803c81e3fd8 [78117.857907] ffff8803c81e3fd8 ffff8803c81e3fd8 ffff8803eecc5ee0 ffff88017912a388 [78117.866841] ffff88017912a240 ffff88017912a268 ffff88017912a390 0000000000000000 [78117.875785] Call Trace: [78117.879135] [] schedule+0x29/0x70 [78117.885303] [] cv_wait_common+0x125/0x150 [spl] [78117.892824] [] ? wake_up_atomic_t+0x30/0x30 [78117.899980] [] __cv_wait+0x15/0x20 [spl] [78117.906853] [] txg_wait_open+0xc3/0x110 [zfs] [78117.914178] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78117.921401] [] dmu_tx_assign+0x91/0x490 [zfs] [78117.928697] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78117.936559] [] ofd_trans_start+0x6b/0xe0 [ofd] [78117.943940] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78117.951927] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78117.959224] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78117.966838] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78117.974593] [] ? update_curr+0x104/0x190 [78117.981371] [] ? __enqueue_entity+0x78/0x80 [78117.988441] [] ? enqueue_entity+0x26c/0xb60 [78117.995532] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [78118.004185] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78118.012444] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78118.021588] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78118.029763] [] ? default_wake_function+0x12/0x20 [78118.037336] [] ? __wake_up_common+0x58/0x90 [78118.044449] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78118.052048] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78118.060886] [] kthread+0xcf/0xe0 [78118.066916] [] ? insert_kthread_work+0x40/0x40 [78118.074314] [] ret_from_fork+0x58/0x90 [78118.080956] [] ? insert_kthread_work+0x40/0x40 [78118.088350] INFO: task ll_ost_io01_025:52856 blocked for more than 120 seconds. [78118.097106] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78118.106449] ll_ost_io01_025 D 0000000000000000 0 52856 2 0x00000080 [78118.114967] ffff8803aee5b830 0000000000000046 ffff8803af36dee0 ffff8803aee5bfd8 [78118.123871] ffff8803aee5bfd8 ffff8803aee5bfd8 ffff8803af36dee0 ffff880823718b88 [78118.132789] ffff880823718a40 ffff880823718a68 ffff880823718b90 0000000000000000 [78118.141704] Call Trace: [78118.145025] [] schedule+0x29/0x70 [78118.151175] [] cv_wait_common+0x125/0x150 [spl] [78118.158687] [] ? wake_up_atomic_t+0x30/0x30 [78118.165806] [] __cv_wait+0x15/0x20 [spl] [78118.172652] [] txg_wait_open+0xc3/0x110 [zfs] [78118.180003] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78118.187225] [] dmu_tx_assign+0x91/0x490 [zfs] [78118.194519] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78118.202400] [] ofd_trans_start+0x6b/0xe0 [ofd] [78118.209799] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78118.217796] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78118.225112] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78118.232715] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78118.240470] [] ? update_curr+0x104/0x190 [78118.247249] [] ? __enqueue_entity+0x78/0x80 [78118.254325] [] ? enqueue_entity+0x26c/0xb60 [78118.261415] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [78118.270091] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78118.278351] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78118.287496] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78118.295644] [] ? default_wake_function+0x12/0x20 [78118.303239] [] ? __wake_up_common+0x58/0x90 [78118.310341] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78118.317966] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78118.326828] [] kthread+0xcf/0xe0 [78118.332891] [] ? insert_kthread_work+0x40/0x40 [78118.340282] [] ret_from_fork+0x58/0x90 [78118.346913] [] ? insert_kthread_work+0x40/0x40 [78118.354334] INFO: task ll_ost_io01_079:58043 blocked for more than 120 seconds. [78118.363117] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [78118.372461] ll_ost_io01_079 D 0000000000000000 0 58043 2 0x00000080 [78118.381006] ffff88006028b830 0000000000000046 ffff88035a609fa0 ffff88006028bfd8 [78118.389956] ffff88006028bfd8 ffff88006028bfd8 ffff88035a609fa0 ffff880823718b88 [78118.398898] ffff880823718a40 ffff880823718a68 ffff880823718b90 0000000000000000 [78118.407831] Call Trace: [78118.411176] [] schedule+0x29/0x70 [78118.417337] [] cv_wait_common+0x125/0x150 [spl] [78118.424861] [] ? wake_up_atomic_t+0x30/0x30 [78118.432006] [] __cv_wait+0x15/0x20 [spl] [78118.438852] [] txg_wait_open+0xc3/0x110 [zfs] [78118.446203] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78118.453423] [] dmu_tx_assign+0x91/0x490 [zfs] [78118.460747] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78118.468616] [] ofd_trans_start+0x6b/0xe0 [ofd] [78118.476026] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78118.483987] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78118.491305] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78118.498886] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78118.506638] [] ? update_curr+0x104/0x190 [78118.513417] [] ? __enqueue_entity+0x78/0x80 [78118.520488] [] ? enqueue_entity+0x26c/0xb60 [78118.527569] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [78118.536240] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78118.544496] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78118.553633] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78118.561781] [] ? default_wake_function+0x12/0x20 [78118.569355] [] ? __wake_up_common+0x58/0x90 [78118.576468] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78118.584092] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [78118.592960] [] kthread+0xcf/0xe0 [78118.599017] [] ? insert_kthread_work+0x40/0x40 [78118.606404] [] ret_from_fork+0x58/0x90 [78118.613019] [] ? insert_kthread_work+0x40/0x40 [78194.380117] LNet: Service thread pid 58043 was inactive for 200.72s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [78194.439407] Pid: 58043, comm: ll_ost_io01_079 [78194.455462] Call Trace: [78194.471542] [] schedule+0x29/0x70 [78194.489390] [] cv_wait_common+0x125/0x150 [spl] [78194.511131] [] ? autoremove_wake_function+0x0/0x40 [78194.533639] [] __cv_wait+0x15/0x20 [spl] [78194.553226] [] txg_wait_open+0xc3/0x110 [zfs] [78194.574163] [] dmu_tx_wait+0x3aa/0x3c0 [zfs] [78194.594770] [] dmu_tx_assign+0x91/0x490 [zfs] [78194.615572] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78194.638031] [] ofd_trans_start+0x6b/0xe0 [ofd] [78194.658996] [] ofd_commitrw_write+0x94b/0x1c50 [ofd] [78194.681626] [] ofd_commitrw+0x4b9/0xac0 [ofd] [78194.702238] [] obd_commitrw+0x2ed/0x330 [ptlrpc] [78194.723696] [] tgt_brw_write+0xff1/0x17c0 [ptlrpc] [78194.745629] [] ? update_curr+0x104/0x190 [78194.764655] [] ? __enqueue_entity+0x78/0x80 [78194.784541] [] ? enqueue_entity+0x26c/0xb60 [78194.804447] [] ? target_bulk_timeout+0x0/0xb0 [ptlrpc] [78194.827507] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78194.850858] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78194.876786] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78194.899836] [] ? default_wake_function+0x12/0x20 [78194.921157] [] ? __wake_up_common+0x58/0x90 [78194.941080] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78194.962438] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [78194.983773] [] kthread+0xcf/0xe0 [78195.000529] [] ? kthread+0x0/0xe0 [78195.017560] [] ret_from_fork+0x58/0x90 [78195.036034] [] ? kthread+0x0/0xe0 [78195.053066] [78195.058645] LustreError: dumping log to /tmp/lustre-log.1510875878.58043 [78196.575549] LNet: Service thread pid 58073 was inactive for 200.40s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [78196.635060] Pid: 58073, comm: ll_ost_io00_085 [78196.651149] Call Trace: [78196.667105] [] schedule+0x29/0x70 [78196.684908] [] cv_wait_common+0x125/0x150 [spl] [78196.706599] [] ? autoremove_wake_function+0x0/0x40 [78196.729051] [] __cv_wait+0x15/0x20 [spl] [78196.748600] [] dmu_tx_wait+0x20b/0x3c0 [zfs] [78196.769240] [] dmu_tx_assign+0x91/0x490 [zfs] [78196.790088] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78196.812570] [] ofd_trans_start+0x6b/0xe0 [ofd] [78196.833564] [] ofd_object_punch+0x70a/0xd20 [ofd] [78196.855344] [] ofd_punch_hdl+0x471/0xaa0 [ofd] [78196.876278] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78196.899732] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78196.925755] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78196.948823] [] ? default_wake_function+0x12/0x20 [78196.970154] [] ? __wake_up_common+0x58/0x90 [78196.990082] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78197.011442] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [78197.032763] [] kthread+0xcf/0xe0 [78197.049513] [] ? kthread+0x0/0xe0 [78197.066544] [] ret_from_fork+0x58/0x90 [78197.085002] [] ? kthread+0x0/0xe0 [78197.102023] [78197.107599] Pid: 30682, comm: ll_ost_out01_00 [78197.122625] Call Trace: [78197.136895] [] schedule+0x29/0x70 [78197.153925] [] cv_wait_common+0x125/0x150 [spl] [78197.174949] [] ? autoremove_wake_function+0x0/0x40 [78197.196860] [] __cv_wait+0x15/0x20 [spl] [78197.215931] [] dmu_tx_wait+0x20b/0x3c0 [zfs] [78197.236136] [] dmu_tx_assign+0x91/0x490 [zfs] [78197.256595] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78197.278799] [] out_tx_end+0x9d/0x5c0 [ptlrpc] [78197.299283] [] out_handle+0x11e7/0x1920 [ptlrpc] [78197.320594] [] ? update_curr+0x104/0x190 [78197.339640] [] ? lustre_msg_get_opc+0x22/0xf0 [ptlrpc] [78197.362706] [] ? tgt_request_preprocess.isra.26+0x299/0x7a0 [ptlrpc] [78197.389784] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78197.413131] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78197.439080] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78197.462131] [] ? default_wake_function+0x12/0x20 [78197.483465] [] ? __wake_up_common+0x58/0x90 [78197.503367] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78197.524653] [] ? __switch_to+0xd7/0x510 [78197.543308] [] ? __schedule+0x350/0x8b0 [78197.561966] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [78197.583167] [] kthread+0xcf/0xe0 [78197.599764] [] ? kthread+0x0/0xe0 [78197.616647] [] ret_from_fork+0x58/0x90 [78197.634978] [] ? kthread+0x0/0xe0 [78197.651855] [78197.657280] LNet: Service thread pid 30681 was inactive for 200.83s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [78197.714247] LNet: Skipped 1 previous similar message [78197.731691] Pid: 30681, comm: ll_ost_out01_00 [78197.747073] Call Trace: [78197.761925] [] schedule+0x29/0x70 [78197.779196] [] cv_wait_common+0x125/0x150 [spl] [78197.800428] [] ? autoremove_wake_function+0x0/0x40 [78197.822480] [] __cv_wait+0x15/0x20 [spl] [78197.841649] [] dmu_tx_wait+0x20b/0x3c0 [zfs] [78197.861954] [] dmu_tx_assign+0x91/0x490 [zfs] [78197.882512] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78197.904767] [] out_tx_end+0x9d/0x5c0 [ptlrpc] [78197.925258] [] out_handle+0x11e7/0x1920 [ptlrpc] [78197.946584] [] ? update_curr+0x104/0x190 [78197.965658] [] ? lustre_msg_get_opc+0x22/0xf0 [ptlrpc] [78197.988751] [] ? tgt_request_preprocess.isra.26+0x299/0x7a0 [ptlrpc] [78198.015856] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78198.039218] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78198.065173] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78198.088266] [] ? default_wake_function+0x12/0x20 [78198.109643] [] ? __wake_up_common+0x58/0x90 [78198.129610] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78198.150971] [] ? __switch_to+0xd7/0x510 [78198.169762] [] ? __schedule+0x350/0x8b0 [78198.188583] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [78198.209935] [] kthread+0xcf/0xe0 [78198.226721] [] ? kthread+0x0/0xe0 [78198.243788] [] ret_from_fork+0x58/0x90 [78198.262293] [] ? kthread+0x0/0xe0 [78198.279362] [78198.284955] Pid: 113259, comm: ll_ost_out01_00 [78198.300303] Call Trace: [78198.314593] [] schedule+0x29/0x70 [78198.331626] [] cv_wait_common+0x125/0x150 [spl] [78198.352660] [] ? autoremove_wake_function+0x0/0x40 [78198.374515] [] __cv_wait+0x15/0x20 [spl] [78198.393532] [] dmu_tx_wait+0x20b/0x3c0 [zfs] [78198.413679] [] dmu_tx_assign+0x91/0x490 [zfs] [78198.434070] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [78198.456188] [] out_tx_end+0x9d/0x5c0 [ptlrpc] [78198.476564] [] out_handle+0x11e7/0x1920 [ptlrpc] [78198.497764] [] ? update_curr+0x104/0x190 [78198.516701] [] ? lustre_msg_get_opc+0x22/0xf0 [ptlrpc] [78198.539658] [] ? tgt_request_preprocess.isra.26+0x299/0x7a0 [ptlrpc] [78198.566623] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [78198.589854] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [78198.615641] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [78198.638548] [] ? default_wake_function+0x12/0x20 [78198.659731] [] ? __wake_up_common+0x58/0x90 [78198.679510] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [78198.700707] [] ? __switch_to+0xd7/0x510 [78198.719334] [] ? __schedule+0x350/0x8b0 [78198.737974] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [78198.759171] [] kthread+0xcf/0xe0 [78198.775785] [] ? kthread+0x0/0xe0 [78198.792683] [] ret_from_fork+0x58/0x90 [78198.811015] [] ? kthread+0x0/0xe0 [78198.827902] [78198.833338] LNet: Service thread pid 113258 was inactive for 202.01s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [78206.667432] LNet: Service thread pid 52541 was inactive for 200.65s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [78206.713214] LNet: Skipped 34 previous similar messages [78206.731803] LustreError: dumping log to /tmp/lustre-log.1510875890.52541 [78208.715389] LNet: Service thread pid 65784 was inactive for 200.62s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [78208.761403] LustreError: dumping log to /tmp/lustre-log.1510875892.65784 [78224.074348] LNet: Service thread pid 58063 was inactive for 200.04s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [78224.120578] LustreError: dumping log to /tmp/lustre-log.1510875907.58063 [78225.610220] LustreError: dumping log to /tmp/lustre-log.1510875909.65785 [78232.777821] LNet: Service thread pid 58034 was inactive for 200.28s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [78232.824278] LNet: Skipped 9 previous similar messages [78232.842963] LustreError: dumping log to /tmp/lustre-log.1510875916.58034 [78298.280809] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [78298.405804] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 3 previous similar messages [78588.996110] Lustre: 58129:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff8806628a8450 x1584194581522992/t0(0) o4->391ce171-6fa3-bf28-8f06-3dd471dea02b@192.168.1.130@o2ib:477/0 lens 608/448 e 23 to 0 dl 1510876277 ref 2 fl Interpret:/0/0 rc 0/0 [78589.508048] Lustre: 39901:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff8807fda4bc50 x1584254470053616/t0(0) o1000->soaked-MDT0000-mdtlov_UUID@192.168.1.108@o2ib:478/0 lens 520/4320 e 23 to 0 dl 1510876278 ref 2 fl Interpret:/0/0 rc 0/0 [78589.549500] Lustre: 39901:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 15 previous similar messages [78590.532028] Lustre: 151724:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff8807ed79f850 x1584254470072064/t0(0) o6->soaked-MDT0000-mdtlov_UUID@192.168.1.108@o2ib:479/0 lens 664/432 e 23 to 0 dl 1510876279 ref 2 fl Interpret:/0/0 rc 0/0 [78590.574675] Lustre: 151724:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 1 previous similar message [78594.685305] Lustre: soaked-OST000b: Client 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) reconnecting [78594.700011] Lustre: soaked-OST000b: Connection restored to 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) [78594.714964] Lustre: Skipped 3 previous similar messages [78595.234683] Lustre: soaked-OST0011: Client f360cbdf-c189-1f9b-11ce-46bac06079cb (at 192.168.1.122@o2ib) reconnecting [78596.239941] Lustre: soaked-OST0011: Client soaked-MDT0000-mdtlov_UUID (at 192.168.1.108@o2ib) reconnecting [78596.254637] Lustre: soaked-OST0011: deleting orphan objects from 0x0:22263180 to 0x0:22276497 [78596.810479] Lustre: soaked-OST000b: deleting orphan objects from 0x0:22306470 to 0x0:22319521 [78600.659394] Lustre: 52537:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff8803821aa050 x1584193796487904/t0(0) o4->60927082-674f-668f-5a1b-304bea3b02dc@192.168.1.121@o2ib:489/0 lens 608/448 e 23 to 0 dl 1510876289 ref 2 fl Interpret:H/0/0 rc 0/0 [78600.704318] Lustre: 52537:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 21 previous similar messages [78607.733705] Lustre: soaked-OST000b: Client 60927082-674f-668f-5a1b-304bea3b02dc (at 192.168.1.121@o2ib) reconnecting [78607.748917] Lustre: Skipped 2 previous similar messages [78618.674287] Lustre: 65771:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff8803821ac050 x1584203740686720/t0(0) o4->ed8fa4bb-d36f-0b40-48fa-fe78100782a5@192.168.1.133@o2ib:507/0 lens 608/448 e 12 to 0 dl 1510876307 ref 2 fl Interpret:/0/0 rc 0/0 [78618.719871] Lustre: 65771:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 1 previous similar message [78625.572400] Lustre: soaked-OST000b: Client ed8fa4bb-d36f-0b40-48fa-fe78100782a5 (at 192.168.1.133@o2ib) reconnecting [78625.587658] Lustre: Skipped 1 previous similar message [78626.750347] Lustre: soaked-OST0011: Connection restored to 7138a171-b124-888f-d923-e084f7df0c22 (at 192.168.1.129@o2ib) [78626.765920] Lustre: Skipped 12 previous similar messages [78627.697736] Lustre: 54287:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff8803821a8c50 x1584203661778528/t0(0) o4->8b0c50cd-9f3a-6240-b7ca-f83ce30a2cb0@192.168.1.128@o2ib:516/0 lens 608/448 e 12 to 0 dl 1510876316 ref 2 fl Interpret:H/0/0 rc 0/0 [78627.743700] Lustre: 54287:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 9 previous similar messages [79030.236145] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 2 seconds [79030.251215] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [79190.671381] Lustre: 52509:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff8808136e4450 x1584194234968000/t0(0) o4->f360cbdf-c189-1f9b-11ce-46bac06079cb@192.168.1.122@o2ib:324/0 lens 608/448 e 0 to 0 dl 1510876879 ref 2 fl Interpret:/2/0 rc 0/0 [79195.695629] Lustre: soaked-OST000b: Client 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) reconnecting [79195.825234] Lustre: Skipped 8 previous similar messages [79195.891417] Lustre: soaked-OST000b: Connection restored to 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) [79196.024050] Lustre: Skipped 1 previous similar message [79197.203545] Lustre: soaked-OST0011: deleting orphan objects from 0x0:22263180 to 0x0:22276529 [79197.876108] Lustre: soaked-OST000b: deleting orphan objects from 0x0:22306470 to 0x0:22319553 [79226.757056] Lustre: soaked-OST000b: Client ed8fa4bb-d36f-0b40-48fa-fe78100782a5 (at 192.168.1.133@o2ib) reconnecting [79226.886657] Lustre: Skipped 6 previous similar messages [79699.195370] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.114@o2ib: 0 seconds [79699.210335] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [79792.746657] Lustre: 52768:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (4/4), not sending early reply req@ffff8806628a8850 x1584194234967920/t0(0) o4->f360cbdf-c189-1f9b-11ce-46bac06079cb@192.168.1.122@o2ib:170/0 lens 608/448 e 0 to 0 dl 1510877480 ref 2 fl Interpret:/2/0 rc 0/0 [79792.854683] Lustre: 52768:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 46 previous similar messages [79794.794532] LNet: Service thread pid 53708 was inactive for 1201.22s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [79794.856862] LNet: Skipped 1 previous similar message [79794.876554] Pid: 53708, comm: ll_ost_io01_049 [79794.893967] Call Trace: [79794.912519] [] schedule+0x29/0x70 [79794.931438] [] rwsem_down_read_failed+0x10d/0x1a0 [79794.954744] [] call_rwsem_down_read_failed+0x18/0x30 [79794.978722] [] down_read+0x20/0x40 [79794.997447] [] osd_read_lock+0x39/0x40 [osd_zfs] [79795.020063] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [79795.044844] [] ofd_preprw+0x422/0x1150 [ofd] [79795.066141] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [79795.088987] [] ? update_curr+0x104/0x190 [79795.108841] [] ? __enqueue_entity+0x78/0x80 [79795.129479] [] ? enqueue_entity+0x26c/0xb60 [79795.150027] [] ? ___slab_alloc+0x209/0x4f0 [79795.170223] [] ? mutex_lock+0x12/0x2f [79795.188924] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [79795.212755] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [79795.239162] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [79795.262653] [] ? default_wake_function+0x12/0x20 [79795.284417] [] ? __wake_up_common+0x58/0x90 [79795.304737] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [79795.326476] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [79795.348163] [] kthread+0xcf/0xe0 [79795.365253] [] ? kthread+0x0/0xe0 [79795.382620] [] ret_from_fork+0x58/0x90 [79795.401374] [] ? kthread+0x0/0xe0 [79795.418688] [79795.424526] LustreError: dumping log to /tmp/lustre-log.1510877479.53708 [79795.512336] LNet: Service thread pid 58129 was inactive for 1201.94s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [79795.570353] Pid: 58129, comm: ll_ost_io01_091 [79795.586846] Call Trace: [79795.603223] [] schedule+0x29/0x70 [79795.621212] [] rwsem_down_read_failed+0x10d/0x1a0 [79795.643693] [] call_rwsem_down_read_failed+0x18/0x30 [79795.666897] [] down_read+0x20/0x40 [79795.684810] [] osd_read_lock+0x39/0x40 [osd_zfs] [79795.706703] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [79795.730783] [] ofd_preprw+0x422/0x1150 [ofd] [79795.751409] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [79795.773591] [] ? update_curr+0x104/0x190 [79795.792853] [] ? __enqueue_entity+0x78/0x80 [79795.812918] [] ? enqueue_entity+0x26c/0xb60 [79795.832926] [] ? mutex_lock+0x12/0x2f [79795.851191] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [79795.874568] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [79795.900499] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [79795.923556] [] ? default_wake_function+0x12/0x20 [79795.944909] [] ? __wake_up_common+0x58/0x90 [79795.964856] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [79795.986235] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [79796.007571] [] kthread+0xcf/0xe0 [79796.024327] [] ? kthread+0x0/0xe0 [79796.041387] [] ret_from_fork+0x58/0x90 [79796.059855] [] ? kthread+0x0/0xe0 [79796.076889] [79796.630032] Lustre: soaked-OST000b: Client 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) reconnecting [79796.666361] Lustre: Skipped 8 previous similar messages [79796.685272] Lustre: soaked-OST000b: Connection restored to 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) [79796.722387] Lustre: Skipped 15 previous similar messages [79797.798253] Lustre: soaked-OST0011: deleting orphan objects from 0x0:22263180 to 0x0:22276561 [79798.798908] Lustre: soaked-OST000b: deleting orphan objects from 0x0:22306470 to 0x0:22319585 [79798.890221] LNet: Service thread pid 21975 was inactive for 1202.70s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [79798.890223] Pid: 21975, comm: ll_ost_out01_00 [79798.890224] Call Trace: [79798.890233] [] schedule+0x29/0x70 [79798.890252] [] cv_wait_common+0x125/0x150 [spl] [79798.890257] [] ? autoremove_wake_function+0x0/0x40 [79798.890263] [] __cv_wait+0x15/0x20 [spl] [79798.890300] [] dmu_tx_wait+0x20b/0x3c0 [zfs] [79798.890326] [] dmu_tx_assign+0x91/0x490 [zfs] [79798.890333] [] osd_trans_start+0xa7/0x3a0 [osd_zfs] [79798.890393] [] out_tx_end+0x9d/0x5c0 [ptlrpc] [79798.890438] [] out_handle+0x11e7/0x1920 [ptlrpc] [79798.890442] [] ? update_curr+0x104/0x190 [79798.890480] [] ? lustre_msg_get_opc+0x22/0xf0 [ptlrpc] [79798.890523] [] ? tgt_request_preprocess.isra.26+0x299/0x7a0 [ptlrpc] [79798.890566] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [79798.890605] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [79798.890643] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [79798.890646] [] ? default_wake_function+0x0/0x20 [79798.890683] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [79798.890687] [] ? __switch_to+0xd7/0x510 [79798.890689] [] ? __schedule+0x350/0x8b0 [79798.890726] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [79798.890728] [] kthread+0xcf/0xe0 [79798.890730] [] ? kthread+0x0/0xe0 [79798.890732] [] ret_from_fork+0x58/0x90 [79798.890734] [] ? kthread+0x0/0xe0 [79798.890734] [79798.890736] LustreError: dumping log to /tmp/lustre-log.1510877482.21975 [79798.928212] Pid: 58001, comm: ll_ost_io01_072 [79798.928213] Call Trace: [79798.928230] [] schedule+0x29/0x70 [79798.928232] [] rwsem_down_read_failed+0x10d/0x1a0 [79798.928237] [] call_rwsem_down_read_failed+0x18/0x30 [79798.928239] [] down_read+0x20/0x40 [79798.928246] [] osd_read_lock+0x39/0x40 [osd_zfs] [79798.928256] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [79798.928262] [] ofd_preprw+0x422/0x1150 [ofd] [79798.928307] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [79798.928309] [] ? __enqueue_entity+0x78/0x80 [79798.928312] [] ? enqueue_entity+0x26c/0xb60 [79798.928315] [] ? mutex_lock+0x12/0x2f [79798.928355] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [79798.928391] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [79798.928426] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [79798.928459] [] ? ptlrpc_server_post_idle_rqbds+0x73/0xf0 [ptlrpc] [79798.928493] [] ? ptlrpc_grow_req_bufs+0x50/0x290 [ptlrpc] [79798.928526] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [79798.928560] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [79798.928562] [] kthread+0xcf/0xe0 [79798.928563] [] ? kthread+0x0/0xe0 [79798.928565] [] ret_from_fork+0x58/0x90 [79798.928566] [] ? kthread+0x0/0xe0 [79798.928567] [79798.928569] Pid: 58014, comm: ll_ost_io01_074 [79798.928570] Call Trace: [79798.928572] [] schedule+0x29/0x70 [79798.928573] [] rwsem_down_read_failed+0x10d/0x1a0 [79798.928576] [] call_rwsem_down_read_failed+0x18/0x30 [79798.928577] [] down_read+0x20/0x40 [79798.928583] [] osd_read_lock+0x39/0x40 [osd_zfs] [79798.928590] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [79798.928596] [] ofd_preprw+0x422/0x1150 [ofd] [79798.928635] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [79798.928638] [] ? update_curr+0x104/0x190 [79798.928640] [] ? update_curr+0x104/0x190 [79798.928642] [] ? __enqueue_entity+0x78/0x80 [79798.928644] [] ? enqueue_entity+0x26c/0xb60 [79798.928647] [] ? mutex_lock+0x12/0x2f [79798.928685] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [79798.928719] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [79798.928752] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [79798.928754] [] ? default_wake_function+0x12/0x20 [79798.928757] [] ? __wake_up_common+0x58/0x90 [79798.928790] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [79798.928824] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [79798.928825] [] kthread+0xcf/0xe0 [79798.928827] [] ? kthread+0x0/0xe0 [79798.928829] [] ret_from_fork+0x58/0x90 [79798.928830] [] ? kthread+0x0/0xe0 [79798.928831] [79798.928833] LNet: Service thread pid 58009 was inactive for 1203.75s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [79811.177514] LNet: Service thread pid 52537 was inactive for 1203.47s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [79811.223363] LNet: Skipped 18 previous similar messages [79811.241832] LustreError: dumping log to /tmp/lustre-log.1510877494.52537 [79827.560518] LNet: Service thread pid 64548 was inactive for 1201.57s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [79827.606551] LNet: Skipped 1 previous similar message [79827.624573] LustreError: dumping log to /tmp/lustre-log.1510877511.64548 [79831.656344] LNet: Service thread pid 53348 was inactive for 1204.35s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [79831.702555] LNet: Skipped 8 previous similar messages [79831.720953] LustreError: dumping log to /tmp/lustre-log.1510877515.53348 [80392.774082] Lustre: 39901:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff880801550450 x1584254470054720/t0(0) o1000->soaked-MDT0000-mdtlov_UUID@192.168.1.108@o2ib:16/0 lens 520/0 e 0 to 0 dl 1510878081 ref 2 fl New:/2/ffffffff rc 0/-1 [80392.874456] Lustre: 39901:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 48 previous similar messages [80396.869806] LNet: Service thread pid 52450 was inactive for 1200.83s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [80396.930260] LNet: Skipped 2 previous similar messages [80396.949103] Pid: 52450, comm: ll_ost_io01_007 [80396.965479] Call Trace: [80396.982149] [] schedule+0x29/0x70 [80397.000245] [] rwsem_down_read_failed+0x10d/0x1a0 [80397.022824] [] call_rwsem_down_read_failed+0x18/0x30 [80397.046154] [] down_read+0x20/0x40 [80397.064260] [] osd_read_lock+0x39/0x40 [osd_zfs] [80397.086285] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80397.110541] [] ofd_preprw+0x422/0x1150 [ofd] [80397.131365] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80397.153813] [] ? mutex_lock+0x12/0x2f [80397.172534] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80397.196342] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80397.222690] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80397.246110] [] ? default_wake_function+0x0/0x20 [80397.267499] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80397.289170] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80397.310808] [] kthread+0xcf/0xe0 [80397.327865] [] ? kthread+0x0/0xe0 [80397.345209] [] ret_from_fork+0x58/0x90 [80397.363987] [] ? kthread+0x0/0xe0 [80397.381343] [80397.387252] LustreError: dumping log to /tmp/lustre-log.1510878081.52450 [80397.455321] Pid: 55993, comm: ll_ost_io01_064 [80397.472080] Call Trace: [80397.489486] [] schedule+0x29/0x70 [80397.507838] [] rwsem_down_read_failed+0x10d/0x1a0 [80397.530605] [] call_rwsem_down_read_failed+0x18/0x30 [80397.554086] [] down_read+0x20/0x40 [80397.572339] [] osd_read_lock+0x39/0x40 [osd_zfs] [80397.594454] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80397.618744] [] ofd_preprw+0x422/0x1150 [ofd] [80397.639592] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80397.661993] [] ? mutex_lock+0x12/0x2f [80397.680669] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80397.704400] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80397.714331] Lustre: soaked-OST000b: Client 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) reconnecting [80397.714332] Lustre: Skipped 15 previous similar messages [80397.714359] Lustre: soaked-OST000b: Connection restored to 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) [80397.714360] Lustre: Skipped 15 previous similar messages [80397.842522] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80397.866593] [] ? default_wake_function+0x0/0x20 [80397.888571] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80397.910734] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80397.932695] [] kthread+0xcf/0xe0 [80397.949954] [] ? kthread+0x0/0xe0 [80397.967425] [] ret_from_fork+0x58/0x90 [80397.986285] [] ? kthread+0x0/0xe0 [80398.003672] [80398.009541] Pid: 58096, comm: ll_ost_io01_088 [80398.024852] Call Trace: [80398.039565] [] schedule+0x29/0x70 [80398.056790] [] rwsem_down_read_failed+0x10d/0x1a0 [80398.078549] [] call_rwsem_down_read_failed+0x18/0x30 [80398.101136] [] down_read+0x20/0x40 [80398.118555] [] osd_read_lock+0x39/0x40 [osd_zfs] [80398.139895] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80398.163597] [] ofd_preprw+0x422/0x1150 [ofd] [80398.183858] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80398.205786] [] ? update_curr+0x104/0x190 [80398.224850] [] ? __enqueue_entity+0x78/0x80 [80398.244767] [] ? enqueue_entity+0x26c/0xb60 [80398.264680] [] ? mutex_lock+0x12/0x2f [80398.282924] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80398.306316] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80398.332293] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80398.355373] [] ? default_wake_function+0x12/0x20 [80398.376746] [] ? __wake_up_common+0x58/0x90 [80398.396713] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80398.418105] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80398.439459] [] kthread+0xcf/0xe0 [80398.456243] [] ? kthread+0x0/0xe0 [80398.473307] [] ret_from_fork+0x58/0x90 [80398.491794] [] ? kthread+0x0/0xe0 [80398.508850] [80398.514448] Pid: 52734, comm: ll_ost_io01_022 [80398.529487] Call Trace: [80398.543789] [] schedule+0x29/0x70 [80398.560827] [] rwsem_down_read_failed+0x10d/0x1a0 [80398.582447] [] call_rwsem_down_read_failed+0x18/0x30 [80398.604921] [] down_read+0x20/0x40 [80398.622245] [] osd_read_lock+0x39/0x40 [osd_zfs] [80398.643577] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80398.667191] [] ofd_preprw+0x422/0x1150 [ofd] [80398.687369] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80398.709185] [] ? __enqueue_entity+0x78/0x80 [80398.720759] Lustre: soaked-OST0011: deleting orphan objects from 0x0:22263180 to 0x0:22276593 [80398.757608] [] ? enqueue_entity+0x26c/0xb60 [80398.777370] [] ? mutex_lock+0x12/0x2f [80398.795438] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80398.818684] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80398.844473] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80398.867375] [] ? default_wake_function+0x12/0x20 [80398.888567] [] ? __wake_up_common+0x58/0x90 [80398.908358] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80398.929552] [] ? __switch_to+0xd7/0x510 [80398.948185] [] ? __schedule+0x350/0x8b0 [80398.966844] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80398.988049] [] kthread+0xcf/0xe0 [80399.004699] [] ? kthread+0x0/0xe0 [80399.021620] [] ret_from_fork+0x58/0x90 [80399.039980] [] ? kthread+0x0/0xe0 [80399.056879] [80399.062321] Pid: 53882, comm: ll_ost_io01_057 [80399.077225] Call Trace: [80399.091250] [] schedule+0x29/0x70 [80399.108164] [] rwsem_down_read_failed+0x10d/0x1a0 [80399.129663] [] call_rwsem_down_read_failed+0x18/0x30 [80399.152024] [] down_read+0x20/0x40 [80399.169221] [] osd_read_lock+0x39/0x40 [osd_zfs] [80399.190410] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80399.213912] [] ofd_preprw+0x422/0x1150 [ofd] [80399.233988] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80399.255766] [] ? update_curr+0x104/0x190 [80399.274693] [] ? __enqueue_entity+0x78/0x80 [80399.294440] [] ? enqueue_entity+0x26c/0xb60 [80399.314203] [] ? mutex_lock+0x12/0x2f [80399.332277] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80399.355498] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80399.381306] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80399.404221] [] ? default_wake_function+0x12/0x20 [80399.425427] [] ? __wake_up_common+0x58/0x90 [80399.445238] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80399.466465] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80399.487714] [] kthread+0xcf/0xe0 [80399.504340] [] ? kthread+0x0/0xe0 [80399.521248] [] ret_from_fork+0x58/0x90 [80399.539599] [] ? kthread+0x0/0xe0 [80399.556505] [80399.561995] LNet: Service thread pid 58032 was inactive for 1203.52s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [80399.605832] LNet: Skipped 1 previous similar message [80399.721685] Lustre: soaked-OST000b: deleting orphan objects from 0x0:22306470 to 0x0:22319617 [80413.252786] LustreError: dumping log to /tmp/lustre-log.1510878096.54287 [80429.635800] LNet: Service thread pid 58033 was inactive for 1202.62s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [80429.681880] LNet: Skipped 14 previous similar messages [80429.700512] LustreError: dumping log to /tmp/lustre-log.1510878113.58033 [80431.150745] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.115@o2ib: 1 seconds [80431.186404] LNet: 21460:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 2 previous similar messages [80655.218509] LustreError: 137-5: soaked-OST0004_UUID: not available for connect from 192.168.1.122@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80655.280626] LustreError: Skipped 3 previous similar messages [80656.453864] LustreError: 137-5: soaked-OST0004_UUID: not available for connect from 192.168.1.108@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80656.516228] LustreError: Skipped 3 previous similar messages [80658.289574] LustreError: 137-5: soaked-OST0004_UUID: not available for connect from 192.168.1.139@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80658.351920] LustreError: Skipped 3 previous similar messages [80661.333449] LustreError: 137-5: soaked-OST0004_UUID: not available for connect from 192.168.1.124@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80661.396348] LustreError: Skipped 11 previous similar messages [80668.044346] LustreError: 137-5: soaked-OST0004_UUID: not available for connect from 192.168.1.134@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80668.107582] LustreError: Skipped 38 previous similar messages [80676.818932] LustreError: 137-5: soaked-OST0010_UUID: not available for connect from 192.168.1.135@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80676.818934] LustreError: 137-5: soaked-OST0016_UUID: not available for connect from 192.168.1.135@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80676.818939] LustreError: Skipped 47 previous similar messages [80676.967266] LustreError: Skipped 2 previous similar messages [80709.234716] LustreError: 137-5: soaked-OST0004_UUID: not available for connect from 192.168.1.123@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80709.298473] LustreError: Skipped 22 previous similar messages [80773.241738] LustreError: 137-5: soaked-OST0004_UUID: not available for connect from 192.168.1.138@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80773.459953] LustreError: Skipped 196 previous similar messages [80885.034821] Lustre: soaked-OST0016: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [80890.426412] Lustre: soaked-OST0016: Will be in recovery for at least 2:30, or until 31 clients reconnect [80902.421568] Lustre: soaked-OST0004: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [80910.372672] Lustre: soaked-OST0004: Will be in recovery for at least 2:30, or until 31 clients reconnect [80910.386485] Lustre: soaked-OST0004: Connection restored to f360cbdf-c189-1f9b-11ce-46bac06079cb (at 192.168.1.122@o2ib) [80910.401573] Lustre: Skipped 32 previous similar messages [80912.108679] LustreError: 137-5: soaked-OST0010_UUID: not available for connect from 192.168.1.126@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [80912.134733] LustreError: Skipped 242 previous similar messages [80918.978762] Lustre: soaked-OST000a: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [80920.088448] Lustre: soaked-OST000a: Will be in recovery for at least 2:30, or until 31 clients reconnect [80941.533158] Lustre: soaked-OST0010: Imperative Recovery enabled, recovery window shrunk from 300-900 down to 150-900 [80942.704986] Lustre: soaked-OST0010: Will be in recovery for at least 2:30, or until 31 clients reconnect [80958.174551] Lustre: soaked-OST0004: Recovery over after 0:47, of 31 clients 31 recovered and 0 were evicted. [80958.241585] Lustre: soaked-OST0004: deleting orphan objects from 0x0:22360909 to 0x0:22367329 [80961.831507] Lustre: soaked-OST000a: Recovery over after 0:42, of 31 clients 31 recovered and 0 were evicted. [80961.871951] Lustre: soaked-OST000a: deleting orphan objects from 0x0:22334083 to 0x0:22345937 [80993.225446] Lustre: 53363:0:(service.c:1356:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/5), not sending early reply req@ffff88072ac36450 x1584194234968112/t0(0) o4->f360cbdf-c189-1f9b-11ce-46bac06079cb@192.168.1.122@o2ib:616/0 lens 608/448 e 0 to 0 dl 1510878681 ref 2 fl Interpret:/2/0 rc 0/0 [80993.271321] Lustre: 53363:0:(service.c:1356:ptlrpc_at_send_early_reply()) Skipped 19 previous similar messages [80998.638565] Lustre: soaked-OST000b: Client 391ce171-6fa3-bf28-8f06-3dd471dea02b (at 192.168.1.130@o2ib) reconnecting [80998.653382] Lustre: Skipped 15 previous similar messages [80998.945064] LNet: Service thread pid 52543 was inactive for 1201.45s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [80998.970165] LNet: Skipped 4 previous similar messages [80998.978513] Pid: 52543, comm: ll_ost_io01_015 [80998.986125] Call Trace: [80998.995355] [] schedule+0x29/0x70 [80999.003334] [] rwsem_down_read_failed+0x10d/0x1a0 [80999.012671] [] call_rwsem_down_read_failed+0x18/0x30 [80999.022307] [] down_read+0x20/0x40 [80999.030022] [] osd_read_lock+0x39/0x40 [osd_zfs] [80999.039133] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80999.048808] [] ofd_preprw+0x422/0x1150 [ofd] [80999.057396] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80999.066318] [] ? __enqueue_entity+0x78/0x80 [80999.074534] [] ? enqueue_entity+0x26c/0xb60 [80999.082656] [] ? mutex_lock+0x12/0x2f [80999.090310] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80999.099511] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80999.109641] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80999.118653] [] ? default_wake_function+0x12/0x20 [80999.127046] [] ? __wake_up_common+0x58/0x90 [80999.134888] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80999.143200] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80999.151407] [] kthread+0xcf/0xe0 [80999.158061] [] ? kthread+0x0/0xe0 [80999.164757] [] ret_from_fork+0x58/0x90 [80999.171900] [] ? kthread+0x0/0xe0 [80999.178494] [80999.181218] LustreError: dumping log to /tmp/lustre-log.1510878682.52543 [80999.536567] Pid: 52471, comm: ll_ost_io01_009 [80999.542420] Call Trace: [80999.550925] [] schedule+0x29/0x70 [80999.558512] [] rwsem_down_read_failed+0x10d/0x1a0 [80999.567545] [] call_rwsem_down_read_failed+0x18/0x30 [80999.576758] [] down_read+0x20/0x40 [80999.584121] [] osd_read_lock+0x39/0x40 [osd_zfs] [80999.592871] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80999.602253] [] ofd_preprw+0x422/0x1150 [ofd] [80999.610406] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80999.619064] [] ? update_curr+0x104/0x190 [80999.626587] [] ? account_entity_dequeue+0xae/0xd0 [80999.634980] [] ? dequeue_entity+0x11c/0x5d0 [80999.642666] [] ? mutex_lock+0x1f/0x2f [80999.649816] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80999.658594] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80999.668256] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80999.676807] [] ? default_wake_function+0x0/0x20 [80999.684703] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80999.692627] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80999.694037] Lustre: soaked-OST0011: deleting orphan objects from 0x0:22263180 to 0x0:22276625 [80999.711092] [] kthread+0xcf/0xe0 [80999.717262] [] ? kthread+0x0/0xe0 [80999.723518] [] ret_from_fork+0x58/0x90 [80999.730249] [] ? kthread+0x0/0xe0 [80999.736487] [80999.738834] Pid: 58244, comm: ll_ost_io01_092 [80999.744388] Call Trace: [80999.750130] [] schedule+0x29/0x70 [80999.756362] [] rwsem_down_read_failed+0x10d/0x1a0 [80999.764145] [] call_rwsem_down_read_failed+0x18/0x30 [80999.772226] [] down_read+0x20/0x40 [80999.778566] [] osd_read_lock+0x39/0x40 [osd_zfs] [80999.786253] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80999.794708] [] ofd_preprw+0x422/0x1150 [ofd] [80999.802047] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80999.809914] [] ? __enqueue_entity+0x78/0x80 [80999.817097] [] ? enqueue_entity+0x26c/0xb60 [80999.824286] [] ? mutex_lock+0x12/0x2f [80999.830916] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [80999.839297] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [80999.848567] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [80999.856849] [] ? default_wake_function+0x12/0x20 [80999.864560] [] ? __wake_up_common+0x58/0x90 [80999.871806] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [80999.879539] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [80999.887226] [] kthread+0xcf/0xe0 [80999.893353] [] ? kthread+0x0/0xe0 [80999.899604] [] ret_from_fork+0x58/0x90 [80999.906316] [] ? kthread+0x0/0xe0 [80999.912558] [80999.914905] Pid: 53531, comm: ll_ost_io01_046 [80999.920461] Call Trace: [80999.926149] [] schedule+0x29/0x70 [80999.932344] [] rwsem_down_read_failed+0x10d/0x1a0 [80999.940107] [] call_rwsem_down_read_failed+0x18/0x30 [80999.948167] [] down_read+0x20/0x40 [80999.954477] [] osd_read_lock+0x39/0x40 [osd_zfs] [80999.962158] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [80999.970627] [] ofd_preprw+0x422/0x1150 [ofd] [80999.977955] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [80999.985814] [] ? update_curr+0x104/0x190 [80999.992697] [] ? update_curr+0x104/0x190 [80999.999587] [] ? __enqueue_entity+0x78/0x80 [81000.006760] [] ? enqueue_entity+0x26c/0xb60 [81000.013945] [] ? mutex_lock+0x12/0x2f [81000.020564] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [81000.028914] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [81000.038133] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [81000.046380] [] ? default_wake_function+0x12/0x20 [81000.054027] [] ? __wake_up_common+0x58/0x90 [81000.061217] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [81000.068901] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [81000.076570] [] kthread+0xcf/0xe0 [81000.082685] [] ? kthread+0x0/0xe0 [81000.088884] [] ret_from_fork+0x58/0x90 [81000.095572] [] ? kthread+0x0/0xe0 [81000.101768] [81000.104077] Pid: 52504, comm: ll_ost_io01_012 [81000.109589] Call Trace: [81000.115197] [] schedule+0x29/0x70 [81000.121358] [] rwsem_down_read_failed+0x10d/0x1a0 [81000.129080] [] call_rwsem_down_read_failed+0x18/0x30 [81000.137113] [] down_read+0x20/0x40 [81000.143407] [] osd_read_lock+0x39/0x40 [osd_zfs] [81000.151039] [] ofd_preprw_write.isra.30+0xc6/0xd80 [ofd] [81000.159471] [] ofd_preprw+0x422/0x1150 [ofd] [81000.166738] [] tgt_brw_write+0xc34/0x17c0 [ptlrpc] [81000.174569] [] ? update_curr+0x104/0x190 [81000.181435] [] ? account_entity_dequeue+0xae/0xd0 [81000.189152] [] ? dequeue_entity+0x11c/0x5d0 [81000.196297] [] ? mutex_lock+0x12/0x2f [81000.202907] [] tgt_request_handle+0x925/0x1370 [ptlrpc] [81000.211223] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [81000.220440] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [81000.228642] [] ? default_wake_function+0x0/0x20 [81000.236162] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [81000.243801] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [81000.251386] [] kthread+0xcf/0xe0 [81000.257426] [] ? kthread+0x0/0xe0 [81000.263527] [] ret_from_fork+0x58/0x90 [81000.270120] [] ? kthread+0x0/0xe0 [81000.276211] [81000.278416] LNet: Service thread pid 52768 was inactive for 1203.60s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [81000.294040] LNet: Skipped 10 previous similar messages [81000.696382] Lustre: soaked-OST000b: deleting orphan objects from 0x0:22306470 to 0x0:22319649 [81011.232352] LustreError: dumping log to /tmp/lustre-log.1510878694.65782 [81015.328416] LustreError: dumping log to /tmp/lustre-log.1510878698.46347 [81031.711120] LustreError: dumping log to /tmp/lustre-log.1510878715.65793 [81040.926632] Lustre: soaked-OST0016: recovery is timed out, evict stale exports [81040.936461] Lustre: soaked-OST0016: disconnecting 1 stale clients [81041.782072] Lustre: soaked-OST0016: Recovery over after 2:31, of 31 clients 30 recovered and 1 was evicted. [81093.147416] Lustre: soaked-OST0010: recovery is timed out, evict stale exports [81093.157352] Lustre: soaked-OST0010: disconnecting 1 stale clients [81093.217742] Lustre: 61117:0:(ldlm_lib.c:1747:extend_recovery_timer()) soaked-OST0010: extended recovery timer reaching hard limit: 900, extend: 1 [81094.620700] Lustre: soaked-OST0010: Recovery over after 2:32, of 31 clients 30 recovered and 1 was evicted. [81094.867857] LustreError: 64553:0:(ofd_io.c:616:ofd_preprw_write()) soaked-OST0010: BRW to missing obj 0x0:22337782 [81094.881368] LustreError: 12092:0:(lustre_dlm.h:1372:ldlm_res_lvbo_update()) delayed lvb init failed (rc -2) [81094.893250] format at lustre_dlm.h:1098:ldlm_lvbo_fill doesn't end in newline [81094.902181] LustreError: 12092:0:(lustre_dlm.h:1098:ldlm_lvbo_fill()) lock ffff8805cf97b400: delayed lvb init failed (rc -2) [81094.914613] LustreError: 12092:0:(ldlm_lock.c:2259:__ldlm_reprocess_all()) ASSERTION( list_empty(&rpc_list) ) failed: [81094.928610] LustreError: 12092:0:(ldlm_lock.c:2259:__ldlm_reprocess_all()) LBUG [81094.937664] Pid: 12092, comm: ldlm_cn01_053 [81094.943227] Call Trace: [81094.949397] [] libcfs_call_trace+0x4e/0x60 [libcfs] [81094.957601] [] lbug_with_loc+0x4c/0xb0 [libcfs] [81094.965446] [] __ldlm_reprocess_all+0x32b/0x3a0 [ptlrpc] [81094.974163] [] ldlm_reprocess_all+0x10/0x20 [ptlrpc] [81094.982475] [] ldlm_request_cancel+0x46d/0x720 [ptlrpc] [81094.991117] [] ldlm_handle_cancel+0xba/0x250 [ptlrpc] [81094.999526] [] ldlm_cancel_handler+0x141/0x490 [ptlrpc] [81095.008151] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [81095.017635] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [81095.026120] [] ? default_wake_function+0x12/0x20 [81095.034048] [] ? __wake_up_common+0x58/0x90 [81095.041506] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [81095.049413] [] ? __switch_to+0xd7/0x510 [81095.056451] [] ? __schedule+0x351/0x8b0 [81095.063511] [] ? ptlrpc_main+0x0/0x1e40 [ptlrpc] [81095.071402] [] kthread+0xcf/0xe0 [81095.077745] [] ? kthread+0x0/0xe0 [81095.084197] [] ret_from_fork+0x58/0x90 [81095.091135] [] ? kthread+0x0/0xe0 [81095.097558] [81095.100122] Kernel panic - not syncing: LBUG [81095.105791] CPU: 31 PID: 12092 Comm: ldlm_cn01_053 Tainted: P OE ------------ 3.10.0-693.5.2.el7_lustre.x86_64 #1 [81095.120330] Hardware name: Intel Corporation S2600GZ ........../S2600GZ, BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013 [81095.133802] ffff880428ef3f00 00000000f3956fc6 ffff880602297b88 ffffffff816a3e51 [81095.143037] ffff880602297c08 ffffffff8169dd14 ffffffff00000008 ffff880602297c18 [81095.152271] ffff880602297bb8 00000000f3956fc6 00000000f3956fc6 ffff88082dbcf8b8 [81095.161472] Call Trace: [81095.165116] [] dump_stack+0x19/0x1b [81095.171788] [] panic+0xe8/0x20d [81095.178060] [] lbug_with_loc+0x64/0xb0 [libcfs] [81095.185913] [] __ldlm_reprocess_all+0x32b/0x3a0 [ptlrpc] [81095.194635] [] ldlm_reprocess_all+0x10/0x20 [ptlrpc] [81095.203013] [] ldlm_request_cancel+0x46d/0x720 [ptlrpc] [81095.211669] [] ldlm_handle_cancel+0xba/0x250 [ptlrpc] [81095.220078] [] ldlm_cancel_handler+0x141/0x490 [ptlrpc] [81095.228706] [] ptlrpc_server_handle_request+0x24e/0xab0 [ptlrpc] [81095.238151] [] ? ptlrpc_wait_event+0x98/0x340 [ptlrpc] [81095.246589] [] ? default_wake_function+0x12/0x20 [81095.254424] [] ? __wake_up_common+0x58/0x90 [81095.261863] [] ptlrpc_main+0xa92/0x1e40 [ptlrpc] [81095.269685] [] ? __switch_to+0xd7/0x510 [81095.276621] [] ? __schedule+0x351/0x8b0 [81095.283548] [] ? ptlrpc_register_service+0xe80/0xe80 [ptlrpc] [81095.292590] [] kthread+0xcf/0xe0 [81095.298811] [] ? insert_kthread_work+0x40/0x40 [81095.306390] [] ret_from_fork+0x58/0x90 [81095.313195] [] ? insert_kthread_work+0x40/0x40