[ 0.000000] Booting Linux on physical CPU 0x0000080000 [0x481fd010] [ 0.000000] Linux version 5.10.0-188.0.0.101.oe2203sp3.aarch64 (root@dc-64g.compass-ci) (gcc_old (GCC) 10.3.1, GNU ld (GNU Binutils) 2.37) #1 SMP Wed Feb 21 13:52:43 CST 2024 [ 0.000000] efi: EFI v2.70 by EDK II [ 0.000000] efi: SMBIOS 3.0=0x2f760000 ACPI 2.0=0x2f7f0000 MEMATTR=0x2d430018 ESRT=0x2d4a9f18 MOKvar=0x2d420000 MEMRESERVE=0x2d431498 [ 0.000000] esrt: Reserving ESRT space from 0x000000002d4a9f18 to 0x000000002d4a9f78. [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x000000002F7F0000 000024 (v02 HISI ) [ 0.000000] ACPI: XSDT 0x000000002F7E0000 0000A4 (v01 HISI HIP08 00000000 01000013) [ 0.000000] ACPI: FACP 0x000000002F2B0000 000114 (v06 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: DSDT 0x000000002F020000 00DA76 (v02 HISI HIP08 00000000 INTL 20181213) [ 0.000000] ACPI: BERT 0x000000002F700000 000030 (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: HEST 0x000000002F6E0000 00058C (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: ERST 0x000000002F6A0000 000230 (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: EINJ 0x000000002F690000 000170 (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: GTDT 0x000000002F290000 00007C (v02 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: SDEI 0x000000002F270000 000030 (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: MCFG 0x000000002F070000 00003C (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: SLIT 0x000000002F060000 00003C (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: SPCR 0x000000002F050000 000050 (v02 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: SRAT 0x000000002F040000 0007D0 (v03 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: APIC 0x000000002F030000 001E6C (v04 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: IORT 0x000000002F010000 001678 (v00 HISI HIP08 00000000 INTL 20181213) [ 0.000000] ACPI: PPTT 0x000000002D480000 0031B0 (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: MPAM 0x000000002D470000 0005C4 (v01 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: SPMI 0x000000002D490000 000041 (v05 HISI HIP08 00000000 HISI 20151124) [ 0.000000] ACPI: SPCR: console: uart,mmio,0x3f00002f8,115200 [ 0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x2080000000-0x3fffffffff] [ 0.000000] ACPI: SRAT: Node 1 PXM 1 [mem 0x4000000000-0x57ffffffff] [ 0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff] [ 0.000000] ACPI: SRAT: Node 2 PXM 2 [mem 0x202000000000-0x203fffffffff] [ 0.000000] ACPI: SRAT: Node 3 PXM 3 [mem 0x204000000000-0x205fffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x3fffffa2c0-0x3fffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x57ffffa2c0-0x57ffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x203fffffa2c0-0x203fffffffff] [ 0.000000] NUMA: NODE_DATA [mem 0x205fbfde22c0-0x205fbfde7fff] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000000000-0x00000000ffffffff] [ 0.000000] DMA32 empty [ 0.000000] Normal [mem 0x0000000100000000-0x0000205fffffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000000000-0x000000000000ffff] [ 0.000000] node 0: [mem 0x0000000000010000-0x000000002d41ffff] [ 0.000000] node 0: [mem 0x000000002d420000-0x000000002d42ffff] [ 0.000000] node 0: [mem 0x000000002d430000-0x000000002f07ffff] [ 0.000000] node 0: [mem 0x000000002f080000-0x000000002f26ffff] [ 0.000000] node 0: [mem 0x000000002f270000-0x000000002f27ffff] [ 0.000000] node 0: [mem 0x000000002f280000-0x000000002f28ffff] [ 0.000000] node 0: [mem 0x000000002f290000-0x000000002f29ffff] [ 0.000000] node 0: [mem 0x000000002f2a0000-0x000000002f2affff] [ 0.000000] node 0: [mem 0x000000002f2b0000-0x000000002f2bffff] [ 0.000000] node 0: [mem 0x000000002f2c0000-0x000000002f39ffff] [ 0.000000] node 0: [mem 0x000000002f3a0000-0x000000002f3effff] [ 0.000000] node 0: [mem 0x000000002f3f0000-0x000000002f48ffff] [ 0.000000] node 0: [mem 0x000000002f490000-0x000000002f4affff] [ 0.000000] node 0: [mem 0x000000002f4b0000-0x000000002f5effff] [ 0.000000] node 0: [mem 0x000000002f5f0000-0x000000002f6affff] [ 0.000000] node 0: [mem 0x000000002f6b0000-0x000000002f6b1fff] [ 0.000000] node 0: [mem 0x000000002f6b2000-0x000000002f6bffff] [ 0.000000] node 0: [mem 0x000000002f6c0000-0x000000002f6c0fff] [ 0.000000] node 0: [mem 0x000000002f6c1000-0x000000002f6cffff] [ 0.000000] node 0: [mem 0x000000002f6d0000-0x000000002f6d1fff] [ 0.000000] node 0: [mem 0x000000002f6d2000-0x000000002f6effff] [ 0.000000] node 0: [mem 0x000000002f6f0000-0x000000002f6f1fff] [ 0.000000] node 0: [mem 0x000000002f6f2000-0x000000002f70ffff] [ 0.000000] node 0: [mem 0x000000002f710000-0x000000002f710fff] [ 0.000000] node 0: [mem 0x000000002f711000-0x000000002f74ffff] [ 0.000000] node 0: [mem 0x000000002f750000-0x000000002f76ffff] [ 0.000000] node 0: [mem 0x000000002f770000-0x000000002f770fff] [ 0.000000] node 0: [mem 0x000000002f771000-0x000000002f771fff] [ 0.000000] node 0: [mem 0x000000002f772000-0x000000002f7fffff] [ 0.000000] node 0: [mem 0x000000002f800000-0x000000002facffff] [ 0.000000] node 0: [mem 0x000000002fad0000-0x000000003ee7ffff] [ 0.000000] node 0: [mem 0x000000003ee80000-0x000000003eeaffff] [ 0.000000] node 0: [mem 0x000000003eeb0000-0x000000003fbfffff] [ 0.000000] node 0: [mem 0x000000003fc00000-0x0000000043ffffff] [ 0.000000] node 0: [mem 0x0000000044030000-0x000000004fffffff] [ 0.000000] node 0: [mem 0x0000000050000000-0x000000007fffffff] [ 0.000000] node 0: [mem 0x0000002080000000-0x0000003fffffffff] [ 0.000000] node 1: [mem 0x0000004000000000-0x00000057ffffffff] [ 0.000000] node 2: [mem 0x0000202000000000-0x0000203fffffffff] [ 0.000000] node 3: [mem 0x0000204000000000-0x0000205fffffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000003fffffffff] [ 0.000000] On node 0 totalpages: 33554384 [ 0.000000] DMA zone: 8192 pages used for memmap [ 0.000000] DMA zone: 0 pages reserved [ 0.000000] DMA zone: 524240 pages, LIFO batch:63 [ 0.000000] Normal zone: 516096 pages used for memmap [ 0.000000] Normal zone: 33030144 pages, LIFO batch:63 [ 0.000000] Initmem setup node 1 [mem 0x0000004000000000-0x00000057ffffffff] [ 0.000000] On node 1 totalpages: 25165824 [ 0.000000] Normal zone: 393216 pages used for memmap [ 0.000000] Normal zone: 25165824 pages, LIFO batch:63 [ 0.000000] Initmem setup node 2 [mem 0x0000202000000000-0x0000203fffffffff] [ 0.000000] On node 2 totalpages: 33554432 [ 0.000000] Normal zone: 524288 pages used for memmap [ 0.000000] Normal zone: 33554432 pages, LIFO batch:63 [ 0.000000] Initmem setup node 3 [mem 0x0000204000000000-0x0000205fffffffff] [ 0.000000] On node 3 totalpages: 33554432 [ 0.000000] Normal zone: 524288 pages used for memmap [ 0.000000] Normal zone: 33554432 pages, LIFO batch:63 [ 0.000000] Reserving 256MB of low memory at 1792MB for crashkernel (low RAM limit: 4096MB) [ 0.000000] Reserving 1024MB of memory at 33946624MB for crashkernel (System RAM: 491519MB) [ 0.000000] psci: probing for conduit method from ACPI. [ 0.000000] psci: PSCIv1.1 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: MIGRATE_INFO_TYPE not supported. [ 0.000000] psci: SMC Calling Convention v1.1 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x80000 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x80100 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x80200 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x80300 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x90000 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x90100 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x90200 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x90300 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa0000 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa0100 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa0200 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xa0300 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb0000 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb0100 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb0200 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xb0300 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc0000 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc0100 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc0200 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xc0300 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd0000 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd0100 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd0200 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xd0300 -> Node 0 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x180000 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x180100 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x180200 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x180300 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x190000 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x190100 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x190200 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x190300 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1a0000 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1a0100 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1a0200 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1a0300 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1b0000 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1b0100 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1b0200 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1b0300 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1c0000 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1c0100 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1c0200 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1c0300 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1d0000 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1d0100 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1d0200 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x1d0300 -> Node 1 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x280000 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x280100 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x280200 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x280300 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x290000 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x290100 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x290200 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x290300 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2a0000 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2a0100 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2a0200 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2a0300 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2b0000 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2b0100 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2b0200 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2b0300 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2c0000 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2c0100 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2c0200 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2c0300 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2d0000 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2d0100 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2d0200 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 2 -> MPIDR 0x2d0300 -> Node 2 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x380000 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x380100 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x380200 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x380300 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x390000 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x390100 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x390200 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x390300 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3a0000 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3a0100 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3a0200 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3a0300 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3b0000 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3b0100 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3b0200 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3b0300 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3c0000 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3c0100 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3c0200 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3c0300 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3d0000 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3d0100 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3d0200 -> Node 3 [ 0.000000] ACPI: NUMA: SRAT: PXM 3 -> MPIDR 0x3d0300 -> Node 3 [ 0.000000] percpu: Embedded 35 pages/cpu s106328 r8192 d28840 u143360 [ 0.000000] pcpu-alloc: s106328 r8192 d28840 u143360 alloc=35*4096 [ 0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07 [ 0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15 [ 0.000000] pcpu-alloc: [0] 16 [0] 17 [0] 18 [0] 19 [0] 20 [0] 21 [0] 22 [0] 23 [ 0.000000] pcpu-alloc: [1] 24 [1] 25 [1] 26 [1] 27 [1] 28 [1] 29 [1] 30 [1] 31 [ 0.000000] pcpu-alloc: [1] 32 [1] 33 [1] 34 [1] 35 [1] 36 [1] 37 [1] 38 [1] 39 [ 0.000000] pcpu-alloc: [1] 40 [1] 41 [1] 42 [1] 43 [1] 44 [1] 45 [1] 46 [1] 47 [ 0.000000] pcpu-alloc: [2] 48 [2] 49 [2] 50 [2] 51 [2] 52 [2] 53 [2] 54 [2] 55 [ 0.000000] pcpu-alloc: [2] 56 [2] 57 [2] 58 [2] 59 [2] 60 [2] 61 [2] 62 [2] 63 [ 0.000000] pcpu-alloc: [2] 64 [2] 65 [2] 66 [2] 67 [2] 68 [2] 69 [2] 70 [2] 71 [ 0.000000] pcpu-alloc: [3] 72 [3] 73 [3] 74 [3] 75 [3] 76 [3] 77 [3] 78 [3] 79 [ 0.000000] pcpu-alloc: [3] 80 [3] 81 [3] 82 [3] 83 [3] 84 [3] 85 [3] 86 [3] 87 [ 0.000000] pcpu-alloc: [3] 88 [3] 89 [3] 90 [3] 91 [3] 92 [3] 93 [3] 94 [3] 95 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] CPU features: detected: GIC system register CPU interface [ 0.000000] CPU features: detected: Virtualization Host Extensions [ 0.000000] CPU features: detected: Hardware dirty bit management [ 0.000000] alternatives: patching kernel code [ 0.000000] Fallback order for Node 0: 0 1 2 3 [ 0.000000] Fallback order for Node 1: 1 0 2 3 [ 0.000000] Fallback order for Node 2: 2 3 0 1 [ 0.000000] Fallback order for Node 3: 3 2 0 1 [ 0.000000] Built 4 zonelists, mobility grouping on. Total pages: 123862992 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-5.10.0-188.0.0.101.oe2203sp3.aarch64 root=/dev/mapper/openeuler-root ro rd.lvm.lv=openeuler/root rd.lvm.lv=openeuler/swap video=VGA-1:640x480-32@60me cgroup_disable=files apparmor=0 crashkernel=1024M,high smmu.bypassdev=0x1000:0x17 smmu.bypassdev=0x1000:0x15 console=tty0 [ 0.000000] cgroup: Disabling files control group subsystem [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] software IO TLB: mapped [mem 0x000000006c000000-0x0000000070000000] (64MB) [ 0.000000] Memory: 493706720K/503316288K available (13504K kernel code, 5448K rwdata, 9584K rodata, 4608K init, 11309K bss, 9609568K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=96, Nodes=4 [ 0.000000] ftrace: allocating 45051 entries in 176 pages [ 0.000000] ftrace: allocated 176 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=4096 to nr_cpu_ids=96. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=96 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode [ 0.000000] GICv3: 640 SPIs implemented [ 0.000000] GICv3: 0 Extended SPIs implemented [ 0.000000] GICv3: Distributor has no Range Selector support [ 0.000000] GICv3: 16 PPIs implemented [ 0.000000] GICv3: GICv4 features: DirectLPI [ 0.000000] GICv3: CPU0: found redistributor 80000 region 0:0x00000000ae100000 [ 0.000000] SRAT: PXM 0 -> ITS 0 -> Node 0 [ 0.000000] SRAT: PXM 2 -> ITS 1 -> Node 2 [ 0.000000] ITS [mem 0x202100000-0x20211ffff] [ 0.000000] ITS@0x0000000202100000: Using ITS number 0 [ 0.000000] ITS@0x0000000202100000: allocated 65536 Devices @2080380000 (flat, esz 8, psz 16K, shr 1) [ 0.000000] ITS@0x0000000202100000: allocated 65536 Virtual CPUs @2080800000 (flat, esz 16, psz 4K, shr 1) [ 0.000000] ITS@0x0000000202100000: allocated 256 Interrupt Collections @208034a000 (flat, esz 16, psz 4K, shr 1) [ 0.000000] ITS [mem 0x200202100000-0x20020211ffff] [ 0.000000] ITS@0x0000200202100000: Using ITS number 1 [ 0.000000] ITS@0x0000200202100000: allocated 65536 Devices @202000080000 (flat, esz 8, psz 16K, shr 1) [ 0.000000] ITS@0x0000200202100000: allocated 65536 Virtual CPUs @202000100000 (flat, esz 16, psz 4K, shr 1) [ 0.000000] ITS@0x0000200202100000: allocated 256 Interrupt Collections @202000001000 (flat, esz 16, psz 4K, shr 1) [ 0.000000] GICv3: using LPI property table @0x0000002080360000 [ 0.000000] ITS: Using DirectLPI for VPE invalidation [ 0.000000] ITS: Enabling GICv4 support [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000002080370000 [ 0.000000] rcu: Offload RCU callbacks from CPUs: (none). [ 0.000000] arch_timer: cp15 timer(s) running at 100.00MHz (phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0x1ffffffffffffff max_cycles: 0x171024e7e0, max_idle_ns: 440795205315 ns [ 0.000001] sched_clock: 57 bits at 100MHz, resolution 10ns, wraps every 4398046511100ns [ 0.000121] Console: colour dummy device 80x25 [ 0.000469] printk: console [tty0] enabled [ 0.000508] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl [ 0.000521] ACPI: Core revision 20200925 [ 0.000683] Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=400000) [ 0.000689] pid_max: default: 98304 minimum: 768 [ 0.000773] LSM: Security Framework initializing [ 0.000789] Yama: becoming mindful. [ 0.000798] SELinux: Initializing. [ 0.018358] Dentry cache hash table entries: 16777216 (order: 15, 134217728 bytes, vmalloc) [ 0.027128] Inode-cache hash table entries: 8388608 (order: 14, 67108864 bytes, vmalloc) [ 0.027457] Mount-cache hash table entries: 262144 (order: 9, 2097152 bytes, vmalloc) [ 0.027740] Mountpoint-cache hash table entries: 262144 (order: 9, 2097152 bytes, vmalloc) [ 0.030241] rcu: Hierarchical SRCU implementation. [ 0.030464] Platform MSI: ITS@0x202100000 domain created [ 0.030470] Platform MSI: ITS@0x200202100000 domain created [ 0.030478] PCI/MSI: ITS@0x202100000 domain created [ 0.030481] PCI/MSI: ITS@0x200202100000 domain created [ 0.030489] Remapping and enabling EFI services. [ 0.032209] smp: Bringing up secondary CPUs ... [ 0.032379] Detected VIPT I-cache on CPU1 [ 0.032387] GICv3: CPU1: found redistributor 80100 region 1:0x00000000ae140000 [ 0.032405] GICv3: CPU1: using allocated LPI pending table @0x0000002080900000 [ 0.032456] CPU1: Booted secondary processor 0x0000080100 [0x481fd010] [ 0.033400] Detected VIPT I-cache on CPU2 [ 0.033405] GICv3: CPU2: found redistributor 80200 region 2:0x00000000ae180000 [ 0.033419] GICv3: CPU2: using allocated LPI pending table @0x0000002080910000 [ 0.033467] CPU2: Booted secondary processor 0x0000080200 [0x481fd010] [ 0.034424] Detected VIPT I-cache on CPU3 [ 0.034429] GICv3: CPU3: found redistributor 80300 region 3:0x00000000ae1c0000 [ 0.034442] GICv3: CPU3: using allocated LPI pending table @0x0000002080920000 [ 0.034488] CPU3: Booted secondary processor 0x0000080300 [0x481fd010] [ 0.035455] Detected VIPT I-cache on CPU4 [ 0.035465] GICv3: CPU4: found redistributor 90000 region 4:0x00000000ae200000 [ 0.035489] GICv3: CPU4: using allocated LPI pending table @0x0000002080930000 [ 0.035542] CPU4: Booted secondary processor 0x0000090000 [0x481fd010] [ 0.036510] Detected VIPT I-cache on CPU5 [ 0.036516] GICv3: CPU5: found redistributor 90100 region 5:0x00000000ae240000 [ 0.036531] GICv3: CPU5: using allocated LPI pending table @0x0000002080940000 [ 0.036578] CPU5: Booted secondary processor 0x0000090100 [0x481fd010] [ 0.037545] Detected VIPT I-cache on CPU6 [ 0.037551] GICv3: CPU6: found redistributor 90200 region 6:0x00000000ae280000 [ 0.037565] GICv3: CPU6: using allocated LPI pending table @0x0000002080950000 [ 0.037613] CPU6: Booted secondary processor 0x0000090200 [0x481fd010] [ 0.038566] Detected VIPT I-cache on CPU7 [ 0.038572] GICv3: CPU7: found redistributor 90300 region 7:0x00000000ae2c0000 [ 0.038586] GICv3: CPU7: using allocated LPI pending table @0x0000002080960000 [ 0.038632] CPU7: Booted secondary processor 0x0000090300 [0x481fd010] [ 0.039585] Detected VIPT I-cache on CPU8 [ 0.039594] GICv3: CPU8: found redistributor a0000 region 8:0x00000000ae300000 [ 0.039614] GICv3: CPU8: using allocated LPI pending table @0x0000002080970000 [ 0.039665] CPU8: Booted secondary processor 0x00000a0000 [0x481fd010] [ 0.040617] Detected VIPT I-cache on CPU9 [ 0.040623] GICv3: CPU9: found redistributor a0100 region 9:0x00000000ae340000 [ 0.040638] GICv3: CPU9: using allocated LPI pending table @0x0000002080980000 [ 0.040686] CPU9: Booted secondary processor 0x00000a0100 [0x481fd010] [ 0.041657] Detected VIPT I-cache on CPU10 [ 0.041663] GICv3: CPU10: found redistributor a0200 region 10:0x00000000ae380000 [ 0.041678] GICv3: CPU10: using allocated LPI pending table @0x0000002080990000 [ 0.041725] CPU10: Booted secondary processor 0x00000a0200 [0x481fd010] [ 0.042675] Detected VIPT I-cache on CPU11 [ 0.042681] GICv3: CPU11: found redistributor a0300 region 11:0x00000000ae3c0000 [ 0.042696] GICv3: CPU11: using allocated LPI pending table @0x00000020809a0000 [ 0.042744] CPU11: Booted secondary processor 0x00000a0300 [0x481fd010] [ 0.043721] Detected VIPT I-cache on CPU12 [ 0.043731] GICv3: CPU12: found redistributor b0000 region 12:0x00000000ae400000 [ 0.043754] GICv3: CPU12: using allocated LPI pending table @0x00000020809b0000 [ 0.043803] CPU12: Booted secondary processor 0x00000b0000 [0x481fd010] [ 0.044753] Detected VIPT I-cache on CPU13 [ 0.044760] GICv3: CPU13: found redistributor b0100 region 13:0x00000000ae440000 [ 0.044775] GICv3: CPU13: using allocated LPI pending table @0x00000020809c0000 [ 0.044821] CPU13: Booted secondary processor 0x00000b0100 [0x481fd010] [ 0.045773] Detected VIPT I-cache on CPU14 [ 0.045781] GICv3: CPU14: found redistributor b0200 region 14:0x00000000ae480000 [ 0.045796] GICv3: CPU14: using allocated LPI pending table @0x00000020809d0000 [ 0.045842] CPU14: Booted secondary processor 0x00000b0200 [0x481fd010] [ 0.046811] Detected VIPT I-cache on CPU15 [ 0.046819] GICv3: CPU15: found redistributor b0300 region 15:0x00000000ae4c0000 [ 0.046834] GICv3: CPU15: using allocated LPI pending table @0x00000020809e0000 [ 0.046881] CPU15: Booted secondary processor 0x00000b0300 [0x481fd010] [ 0.047860] Detected VIPT I-cache on CPU16 [ 0.047871] GICv3: CPU16: found redistributor c0000 region 16:0x00000000ae500000 [ 0.047892] GICv3: CPU16: using allocated LPI pending table @0x00000020809f0000 [ 0.047940] CPU16: Booted secondary processor 0x00000c0000 [0x481fd010] [ 0.048898] Detected VIPT I-cache on CPU17 [ 0.048907] GICv3: CPU17: found redistributor c0100 region 17:0x00000000ae540000 [ 0.048922] GICv3: CPU17: using allocated LPI pending table @0x0000002080a00000 [ 0.048967] CPU17: Booted secondary processor 0x00000c0100 [0x481fd010] [ 0.049917] Detected VIPT I-cache on CPU18 [ 0.049925] GICv3: CPU18: found redistributor c0200 region 18:0x00000000ae580000 [ 0.049940] GICv3: CPU18: using allocated LPI pending table @0x0000002080a10000 [ 0.049987] CPU18: Booted secondary processor 0x00000c0200 [0x481fd010] [ 0.050952] Detected VIPT I-cache on CPU19 [ 0.050961] GICv3: CPU19: found redistributor c0300 region 19:0x00000000ae5c0000 [ 0.050975] GICv3: CPU19: using allocated LPI pending table @0x0000002080a20000 [ 0.051021] CPU19: Booted secondary processor 0x00000c0300 [0x481fd010] [ 0.051991] Detected VIPT I-cache on CPU20 [ 0.052003] GICv3: CPU20: found redistributor d0000 region 20:0x00000000ae600000 [ 0.052025] GICv3: CPU20: using allocated LPI pending table @0x0000002080a30000 [ 0.052075] CPU20: Booted secondary processor 0x00000d0000 [0x481fd010] [ 0.053025] Detected VIPT I-cache on CPU21 [ 0.053034] GICv3: CPU21: found redistributor d0100 region 21:0x00000000ae640000 [ 0.053049] GICv3: CPU21: using allocated LPI pending table @0x0000002080a40000 [ 0.053097] CPU21: Booted secondary processor 0x00000d0100 [0x481fd010] [ 0.054044] Detected VIPT I-cache on CPU22 [ 0.054053] GICv3: CPU22: found redistributor d0200 region 22:0x00000000ae680000 [ 0.054068] GICv3: CPU22: using allocated LPI pending table @0x0000002080a50000 [ 0.054115] CPU22: Booted secondary processor 0x00000d0200 [0x481fd010] [ 0.055070] Detected VIPT I-cache on CPU23 [ 0.055080] GICv3: CPU23: found redistributor d0300 region 23:0x00000000ae6c0000 [ 0.055095] GICv3: CPU23: using allocated LPI pending table @0x0000002080a60000 [ 0.055141] CPU23: Booted secondary processor 0x00000d0300 [0x481fd010] [ 0.056116] Detected VIPT I-cache on CPU24 [ 0.056133] GICv3: CPU24: found redistributor 180000 region 24:0x00000000aa100000 [ 0.056154] GICv3: CPU24: using allocated LPI pending table @0x0000002080a70000 [ 0.056220] CPU24: Booted secondary processor 0x0000180000 [0x481fd010] [ 0.057240] Detected VIPT I-cache on CPU25 [ 0.057250] GICv3: CPU25: found redistributor 180100 region 25:0x00000000aa140000 [ 0.057267] GICv3: CPU25: using allocated LPI pending table @0x0000002080a80000 [ 0.057325] CPU25: Booted secondary processor 0x0000180100 [0x481fd010] [ 0.058267] Detected VIPT I-cache on CPU26 [ 0.058278] GICv3: CPU26: found redistributor 180200 region 26:0x00000000aa180000 [ 0.058294] GICv3: CPU26: using allocated LPI pending table @0x0000002080a90000 [ 0.058350] CPU26: Booted secondary processor 0x0000180200 [0x481fd010] [ 0.059313] Detected VIPT I-cache on CPU27 [ 0.059324] GICv3: CPU27: found redistributor 180300 region 27:0x00000000aa1c0000 [ 0.059339] GICv3: CPU27: using allocated LPI pending table @0x0000002080aa0000 [ 0.059395] CPU27: Booted secondary processor 0x0000180300 [0x481fd010] [ 0.060358] Detected VIPT I-cache on CPU28 [ 0.060376] GICv3: CPU28: found redistributor 190000 region 28:0x00000000aa200000 [ 0.060410] GICv3: CPU28: using allocated LPI pending table @0x0000002080ab0000 [ 0.060473] CPU28: Booted secondary processor 0x0000190000 [0x481fd010] [ 0.061413] Detected VIPT I-cache on CPU29 [ 0.061425] GICv3: CPU29: found redistributor 190100 region 29:0x00000000aa240000 [ 0.061443] GICv3: CPU29: using allocated LPI pending table @0x0000002080ac0000 [ 0.061503] CPU29: Booted secondary processor 0x0000190100 [0x481fd010] [ 0.062459] Detected VIPT I-cache on CPU30 [ 0.062471] GICv3: CPU30: found redistributor 190200 region 30:0x00000000aa280000 [ 0.062487] GICv3: CPU30: using allocated LPI pending table @0x0000002080ad0000 [ 0.062546] CPU30: Booted secondary processor 0x0000190200 [0x481fd010] [ 0.063491] Detected VIPT I-cache on CPU31 [ 0.063503] GICv3: CPU31: found redistributor 190300 region 31:0x00000000aa2c0000 [ 0.063519] GICv3: CPU31: using allocated LPI pending table @0x0000002080ae0000 [ 0.063578] CPU31: Booted secondary processor 0x0000190300 [0x481fd010] [ 0.064538] Detected VIPT I-cache on CPU32 [ 0.064555] GICv3: CPU32: found redistributor 1a0000 region 32:0x00000000aa300000 [ 0.064581] GICv3: CPU32: using allocated LPI pending table @0x0000002080af0000 [ 0.064642] CPU32: Booted secondary processor 0x00001a0000 [0x481fd010] [ 0.065603] Detected VIPT I-cache on CPU33 [ 0.065616] GICv3: CPU33: found redistributor 1a0100 region 33:0x00000000aa340000 [ 0.065633] GICv3: CPU33: using allocated LPI pending table @0x0000002080b00000 [ 0.065689] CPU33: Booted secondary processor 0x00001a0100 [0x481fd010] [ 0.066630] Detected VIPT I-cache on CPU34 [ 0.066643] GICv3: CPU34: found redistributor 1a0200 region 34:0x00000000aa380000 [ 0.066659] GICv3: CPU34: using allocated LPI pending table @0x0000002080b10000 [ 0.066715] CPU34: Booted secondary processor 0x00001a0200 [0x481fd010] [ 0.067676] Detected VIPT I-cache on CPU35 [ 0.067689] GICv3: CPU35: found redistributor 1a0300 region 35:0x00000000aa3c0000 [ 0.067706] GICv3: CPU35: using allocated LPI pending table @0x0000002080b20000 [ 0.067762] CPU35: Booted secondary processor 0x00001a0300 [0x481fd010] [ 0.068719] Detected VIPT I-cache on CPU36 [ 0.068738] GICv3: CPU36: found redistributor 1b0000 region 36:0x00000000aa400000 [ 0.068764] GICv3: CPU36: using allocated LPI pending table @0x0000002080b30000 [ 0.068825] CPU36: Booted secondary processor 0x00001b0000 [0x481fd010] [ 0.069785] Detected VIPT I-cache on CPU37 [ 0.069799] GICv3: CPU37: found redistributor 1b0100 region 37:0x00000000aa440000 [ 0.069815] GICv3: CPU37: using allocated LPI pending table @0x0000002080b40000 [ 0.069872] CPU37: Booted secondary processor 0x00001b0100 [0x481fd010] [ 0.070831] Detected VIPT I-cache on CPU38 [ 0.070844] GICv3: CPU38: found redistributor 1b0200 region 38:0x00000000aa480000 [ 0.070861] GICv3: CPU38: using allocated LPI pending table @0x0000002080b50000 [ 0.070917] CPU38: Booted secondary processor 0x00001b0200 [0x481fd010] [ 0.071864] Detected VIPT I-cache on CPU39 [ 0.071878] GICv3: CPU39: found redistributor 1b0300 region 39:0x00000000aa4c0000 [ 0.071895] GICv3: CPU39: using allocated LPI pending table @0x0000002080b60000 [ 0.071953] CPU39: Booted secondary processor 0x00001b0300 [0x481fd010] [ 0.072917] Detected VIPT I-cache on CPU40 [ 0.072936] GICv3: CPU40: found redistributor 1c0000 region 40:0x00000000aa500000 [ 0.072963] GICv3: CPU40: using allocated LPI pending table @0x0000002080b70000 [ 0.073027] CPU40: Booted secondary processor 0x00001c0000 [0x481fd010] [ 0.073964] Detected VIPT I-cache on CPU41 [ 0.073979] GICv3: CPU41: found redistributor 1c0100 region 41:0x00000000aa540000 [ 0.073995] GICv3: CPU41: using allocated LPI pending table @0x0000002080b80000 [ 0.074052] CPU41: Booted secondary processor 0x00001c0100 [0x481fd010] [ 0.074994] Detected VIPT I-cache on CPU42 [ 0.075008] GICv3: CPU42: found redistributor 1c0200 region 42:0x00000000aa580000 [ 0.075025] GICv3: CPU42: using allocated LPI pending table @0x0000002080b90000 [ 0.075080] CPU42: Booted secondary processor 0x00001c0200 [0x481fd010] [ 0.076023] Detected VIPT I-cache on CPU43 [ 0.076038] GICv3: CPU43: found redistributor 1c0300 region 43:0x00000000aa5c0000 [ 0.076054] GICv3: CPU43: using allocated LPI pending table @0x0000002080ba0000 [ 0.076112] CPU43: Booted secondary processor 0x00001c0300 [0x481fd010] [ 0.077070] Detected VIPT I-cache on CPU44 [ 0.077090] GICv3: CPU44: found redistributor 1d0000 region 44:0x00000000aa600000 [ 0.077118] GICv3: CPU44: using allocated LPI pending table @0x0000002080bb0000 [ 0.077180] CPU44: Booted secondary processor 0x00001d0000 [0x481fd010] [ 0.078115] Detected VIPT I-cache on CPU45 [ 0.078131] GICv3: CPU45: found redistributor 1d0100 region 45:0x00000000aa640000 [ 0.078148] GICv3: CPU45: using allocated LPI pending table @0x0000002080bc0000 [ 0.078205] CPU45: Booted secondary processor 0x00001d0100 [0x481fd010] [ 0.079161] Detected VIPT I-cache on CPU46 [ 0.079176] GICv3: CPU46: found redistributor 1d0200 region 46:0x00000000aa680000 [ 0.079193] GICv3: CPU46: using allocated LPI pending table @0x0000002080bd0000 [ 0.079250] CPU46: Booted secondary processor 0x00001d0200 [0x481fd010] [ 0.080192] Detected VIPT I-cache on CPU47 [ 0.080208] GICv3: CPU47: found redistributor 1d0300 region 47:0x00000000aa6c0000 [ 0.080225] GICv3: CPU47: using allocated LPI pending table @0x0000002080be0000 [ 0.080282] CPU47: Booted secondary processor 0x00001d0300 [0x481fd010] [ 0.081307] Detected VIPT I-cache on CPU48 [ 0.081350] GICv3: CPU48: found redistributor 280000 region 48:0x00002000ae100000 [ 0.081412] GICv3: CPU48: using allocated LPI pending table @0x0000002080bf0000 [ 0.081521] CPU48: Booted secondary processor 0x0000280000 [0x481fd010] [ 0.082476] Detected VIPT I-cache on CPU49 [ 0.082505] GICv3: CPU49: found redistributor 280100 region 49:0x00002000ae140000 [ 0.082524] GICv3: CPU49: using allocated LPI pending table @0x0000002080c00000 [ 0.082609] CPU49: Booted secondary processor 0x0000280100 [0x481fd010] [ 0.083538] Detected VIPT I-cache on CPU50 [ 0.083567] GICv3: CPU50: found redistributor 280200 region 50:0x00002000ae180000 [ 0.083585] GICv3: CPU50: using allocated LPI pending table @0x0000002080c10000 [ 0.083670] CPU50: Booted secondary processor 0x0000280200 [0x481fd010] [ 0.084609] Detected VIPT I-cache on CPU51 [ 0.084637] GICv3: CPU51: found redistributor 280300 region 51:0x00002000ae1c0000 [ 0.084656] GICv3: CPU51: using allocated LPI pending table @0x0000002080c20000 [ 0.084740] CPU51: Booted secondary processor 0x0000280300 [0x481fd010] [ 0.085681] Detected VIPT I-cache on CPU52 [ 0.085713] GICv3: CPU52: found redistributor 290000 region 52:0x00002000ae200000 [ 0.085739] GICv3: CPU52: using allocated LPI pending table @0x0000002080c30000 [ 0.085827] CPU52: Booted secondary processor 0x0000290000 [0x481fd010] [ 0.086760] Detected VIPT I-cache on CPU53 [ 0.086789] GICv3: CPU53: found redistributor 290100 region 53:0x00002000ae240000 [ 0.086808] GICv3: CPU53: using allocated LPI pending table @0x0000002080c40000 [ 0.086894] CPU53: Booted secondary processor 0x0000290100 [0x481fd010] [ 0.087811] Detected VIPT I-cache on CPU54 [ 0.087840] GICv3: CPU54: found redistributor 290200 region 54:0x00002000ae280000 [ 0.087860] GICv3: CPU54: using allocated LPI pending table @0x0000002080c50000 [ 0.087946] CPU54: Booted secondary processor 0x0000290200 [0x481fd010] [ 0.088871] Detected VIPT I-cache on CPU55 [ 0.088901] GICv3: CPU55: found redistributor 290300 region 55:0x00002000ae2c0000 [ 0.088920] GICv3: CPU55: using allocated LPI pending table @0x0000002080c60000 [ 0.089005] CPU55: Booted secondary processor 0x0000290300 [0x481fd010] [ 0.089939] Detected VIPT I-cache on CPU56 [ 0.089972] GICv3: CPU56: found redistributor 2a0000 region 56:0x00002000ae300000 [ 0.090001] GICv3: CPU56: using allocated LPI pending table @0x0000002080c70000 [ 0.090087] CPU56: Booted secondary processor 0x00002a0000 [0x481fd010] [ 0.091004] Detected VIPT I-cache on CPU57 [ 0.091034] GICv3: CPU57: found redistributor 2a0100 region 57:0x00002000ae340000 [ 0.091052] GICv3: CPU57: using allocated LPI pending table @0x0000002080c80000 [ 0.091137] CPU57: Booted secondary processor 0x00002a0100 [0x481fd010] [ 0.092061] Detected VIPT I-cache on CPU58 [ 0.092092] GICv3: CPU58: found redistributor 2a0200 region 58:0x00002000ae380000 [ 0.092111] GICv3: CPU58: using allocated LPI pending table @0x0000002080c90000 [ 0.092196] CPU58: Booted secondary processor 0x00002a0200 [0x481fd010] [ 0.093114] Detected VIPT I-cache on CPU59 [ 0.093144] GICv3: CPU59: found redistributor 2a0300 region 59:0x00002000ae3c0000 [ 0.093163] GICv3: CPU59: using allocated LPI pending table @0x0000002080ca0000 [ 0.093247] CPU59: Booted secondary processor 0x00002a0300 [0x481fd010] [ 0.094194] Detected VIPT I-cache on CPU60 [ 0.094228] GICv3: CPU60: found redistributor 2b0000 region 60:0x00002000ae400000 [ 0.094256] GICv3: CPU60: using allocated LPI pending table @0x0000002080cb0000 [ 0.094343] CPU60: Booted secondary processor 0x00002b0000 [0x481fd010] [ 0.095268] Detected VIPT I-cache on CPU61 [ 0.095299] GICv3: CPU61: found redistributor 2b0100 region 61:0x00002000ae440000 [ 0.095318] GICv3: CPU61: using allocated LPI pending table @0x0000002080cc0000 [ 0.095402] CPU61: Booted secondary processor 0x00002b0100 [0x481fd010] [ 0.096328] Detected VIPT I-cache on CPU62 [ 0.096359] GICv3: CPU62: found redistributor 2b0200 region 62:0x00002000ae480000 [ 0.096379] GICv3: CPU62: using allocated LPI pending table @0x0000002080cd0000 [ 0.096462] CPU62: Booted secondary processor 0x00002b0200 [0x481fd010] [ 0.097392] Detected VIPT I-cache on CPU63 [ 0.097424] GICv3: CPU63: found redistributor 2b0300 region 63:0x00002000ae4c0000 [ 0.097443] GICv3: CPU63: using allocated LPI pending table @0x0000002080ce0000 [ 0.097529] CPU63: Booted secondary processor 0x00002b0300 [0x481fd010] [ 0.098462] Detected VIPT I-cache on CPU64 [ 0.098495] GICv3: CPU64: found redistributor 2c0000 region 64:0x00002000ae500000 [ 0.098525] GICv3: CPU64: using allocated LPI pending table @0x0000002080cf0000 [ 0.098612] CPU64: Booted secondary processor 0x00002c0000 [0x481fd010] [ 0.099544] Detected VIPT I-cache on CPU65 [ 0.099575] GICv3: CPU65: found redistributor 2c0100 region 65:0x00002000ae540000 [ 0.099596] GICv3: CPU65: using allocated LPI pending table @0x0000002080d00000 [ 0.099681] CPU65: Booted secondary processor 0x00002c0100 [0x481fd010] [ 0.100603] Detected VIPT I-cache on CPU66 [ 0.100634] GICv3: CPU66: found redistributor 2c0200 region 66:0x00002000ae580000 [ 0.100655] GICv3: CPU66: using allocated LPI pending table @0x0000002080d10000 [ 0.100740] CPU66: Booted secondary processor 0x00002c0200 [0x481fd010] [ 0.101670] Detected VIPT I-cache on CPU67 [ 0.101701] GICv3: CPU67: found redistributor 2c0300 region 67:0x00002000ae5c0000 [ 0.101721] GICv3: CPU67: using allocated LPI pending table @0x0000002080d20000 [ 0.101805] CPU67: Booted secondary processor 0x00002c0300 [0x481fd010] [ 0.102749] Detected VIPT I-cache on CPU68 [ 0.102783] GICv3: CPU68: found redistributor 2d0000 region 68:0x00002000ae600000 [ 0.102811] GICv3: CPU68: using allocated LPI pending table @0x0000002080d30000 [ 0.102898] CPU68: Booted secondary processor 0x00002d0000 [0x481fd010] [ 0.103813] Detected VIPT I-cache on CPU69 [ 0.103845] GICv3: CPU69: found redistributor 2d0100 region 69:0x00002000ae640000 [ 0.103865] GICv3: CPU69: using allocated LPI pending table @0x0000002080d40000 [ 0.103949] CPU69: Booted secondary processor 0x00002d0100 [0x481fd010] [ 0.104883] Detected VIPT I-cache on CPU70 [ 0.104916] GICv3: CPU70: found redistributor 2d0200 region 70:0x00002000ae680000 [ 0.104935] GICv3: CPU70: using allocated LPI pending table @0x0000002080d50000 [ 0.105020] CPU70: Booted secondary processor 0x00002d0200 [0x481fd010] [ 0.105949] Detected VIPT I-cache on CPU71 [ 0.105982] GICv3: CPU71: found redistributor 2d0300 region 71:0x00002000ae6c0000 [ 0.106002] GICv3: CPU71: using allocated LPI pending table @0x0000002080d60000 [ 0.106085] CPU71: Booted secondary processor 0x00002d0300 [0x481fd010] [ 0.107074] Detected VIPT I-cache on CPU72 [ 0.107140] GICv3: CPU72: found redistributor 380000 region 72:0x00002000aa100000 [ 0.107215] GICv3: CPU72: using allocated LPI pending table @0x0000002080d70000 [ 0.107332] CPU72: Booted secondary processor 0x0000380000 [0x481fd010] [ 0.108267] Detected VIPT I-cache on CPU73 [ 0.108305] GICv3: CPU73: found redistributor 380100 region 73:0x00002000aa140000 [ 0.108327] GICv3: CPU73: using allocated LPI pending table @0x0000002080d80000 [ 0.108421] CPU73: Booted secondary processor 0x0000380100 [0x481fd010] [ 0.109352] Detected VIPT I-cache on CPU74 [ 0.109389] GICv3: CPU74: found redistributor 380200 region 74:0x00002000aa180000 [ 0.109410] GICv3: CPU74: using allocated LPI pending table @0x0000002080d90000 [ 0.109505] CPU74: Booted secondary processor 0x0000380200 [0x481fd010] [ 0.110423] Detected VIPT I-cache on CPU75 [ 0.110460] GICv3: CPU75: found redistributor 380300 region 75:0x00002000aa1c0000 [ 0.110481] GICv3: CPU75: using allocated LPI pending table @0x0000002080da0000 [ 0.110576] CPU75: Booted secondary processor 0x0000380300 [0x481fd010] [ 0.111486] Detected VIPT I-cache on CPU76 [ 0.111527] GICv3: CPU76: found redistributor 390000 region 76:0x00002000aa200000 [ 0.111558] GICv3: CPU76: using allocated LPI pending table @0x0000002080db0000 [ 0.111655] CPU76: Booted secondary processor 0x0000390000 [0x481fd010] [ 0.112579] Detected VIPT I-cache on CPU77 [ 0.112617] GICv3: CPU77: found redistributor 390100 region 77:0x00002000aa240000 [ 0.112638] GICv3: CPU77: using allocated LPI pending table @0x0000002080dc0000 [ 0.112732] CPU77: Booted secondary processor 0x0000390100 [0x481fd010] [ 0.113644] Detected VIPT I-cache on CPU78 [ 0.113682] GICv3: CPU78: found redistributor 390200 region 78:0x00002000aa280000 [ 0.113704] GICv3: CPU78: using allocated LPI pending table @0x0000002080dd0000 [ 0.113797] CPU78: Booted secondary processor 0x0000390200 [0x481fd010] [ 0.114712] Detected VIPT I-cache on CPU79 [ 0.114751] GICv3: CPU79: found redistributor 390300 region 79:0x00002000aa2c0000 [ 0.114773] GICv3: CPU79: using allocated LPI pending table @0x0000002080de0000 [ 0.114865] CPU79: Booted secondary processor 0x0000390300 [0x481fd010] [ 0.115801] Detected VIPT I-cache on CPU80 [ 0.115842] GICv3: CPU80: found redistributor 3a0000 region 80:0x00002000aa300000 [ 0.115872] GICv3: CPU80: using allocated LPI pending table @0x0000002080df0000 [ 0.115968] CPU80: Booted secondary processor 0x00003a0000 [0x481fd010] [ 0.116888] Detected VIPT I-cache on CPU81 [ 0.116927] GICv3: CPU81: found redistributor 3a0100 region 81:0x00002000aa340000 [ 0.116947] GICv3: CPU81: using allocated LPI pending table @0x0000002080e00000 [ 0.117041] CPU81: Booted secondary processor 0x00003a0100 [0x481fd010] [ 0.117950] Detected VIPT I-cache on CPU82 [ 0.117990] GICv3: CPU82: found redistributor 3a0200 region 82:0x00002000aa380000 [ 0.118011] GICv3: CPU82: using allocated LPI pending table @0x0000002080e10000 [ 0.118104] CPU82: Booted secondary processor 0x00003a0200 [0x481fd010] [ 0.119020] Detected VIPT I-cache on CPU83 [ 0.119059] GICv3: CPU83: found redistributor 3a0300 region 83:0x00002000aa3c0000 [ 0.119081] GICv3: CPU83: using allocated LPI pending table @0x0000002080e20000 [ 0.119174] CPU83: Booted secondary processor 0x00003a0300 [0x481fd010] [ 0.120103] Detected VIPT I-cache on CPU84 [ 0.120145] GICv3: CPU84: found redistributor 3b0000 region 84:0x00002000aa400000 [ 0.120176] GICv3: CPU84: using allocated LPI pending table @0x0000002080e30000 [ 0.120273] CPU84: Booted secondary processor 0x00003b0000 [0x481fd010] [ 0.121186] Detected VIPT I-cache on CPU85 [ 0.121227] GICv3: CPU85: found redistributor 3b0100 region 85:0x00002000aa440000 [ 0.121249] GICv3: CPU85: using allocated LPI pending table @0x0000002080e40000 [ 0.121343] CPU85: Booted secondary processor 0x00003b0100 [0x481fd010] [ 0.122262] Detected VIPT I-cache on CPU86 [ 0.122303] GICv3: CPU86: found redistributor 3b0200 region 86:0x00002000aa480000 [ 0.122324] GICv3: CPU86: using allocated LPI pending table @0x0000002080e50000 [ 0.122418] CPU86: Booted secondary processor 0x00003b0200 [0x481fd010] [ 0.123337] Detected VIPT I-cache on CPU87 [ 0.123378] GICv3: CPU87: found redistributor 3b0300 region 87:0x00002000aa4c0000 [ 0.123400] GICv3: CPU87: using allocated LPI pending table @0x0000002080e60000 [ 0.123494] CPU87: Booted secondary processor 0x00003b0300 [0x481fd010] [ 0.124406] Detected VIPT I-cache on CPU88 [ 0.124448] GICv3: CPU88: found redistributor 3c0000 region 88:0x00002000aa500000 [ 0.124478] GICv3: CPU88: using allocated LPI pending table @0x0000002080e70000 [ 0.124575] CPU88: Booted secondary processor 0x00003c0000 [0x481fd010] [ 0.125487] Detected VIPT I-cache on CPU89 [ 0.125527] GICv3: CPU89: found redistributor 3c0100 region 89:0x00002000aa540000 [ 0.125549] GICv3: CPU89: using allocated LPI pending table @0x0000002080e80000 [ 0.125643] CPU89: Booted secondary processor 0x00003c0100 [0x481fd010] [ 0.126565] Detected VIPT I-cache on CPU90 [ 0.126606] GICv3: CPU90: found redistributor 3c0200 region 90:0x00002000aa580000 [ 0.126627] GICv3: CPU90: using allocated LPI pending table @0x0000002080e90000 [ 0.126722] CPU90: Booted secondary processor 0x00003c0200 [0x481fd010] [ 0.127628] Detected VIPT I-cache on CPU91 [ 0.127668] GICv3: CPU91: found redistributor 3c0300 region 91:0x00002000aa5c0000 [ 0.127690] GICv3: CPU91: using allocated LPI pending table @0x0000002080ea0000 [ 0.127784] CPU91: Booted secondary processor 0x00003c0300 [0x481fd010] [ 0.128707] Detected VIPT I-cache on CPU92 [ 0.128751] GICv3: CPU92: found redistributor 3d0000 region 92:0x00002000aa600000 [ 0.128781] GICv3: CPU92: using allocated LPI pending table @0x0000002080eb0000 [ 0.128877] CPU92: Booted secondary processor 0x00003d0000 [0x481fd010] [ 0.129805] Detected VIPT I-cache on CPU93 [ 0.129845] GICv3: CPU93: found redistributor 3d0100 region 93:0x00002000aa640000 [ 0.129867] GICv3: CPU93: using allocated LPI pending table @0x0000002080ec0000 [ 0.129963] CPU93: Booted secondary processor 0x00003d0100 [0x481fd010] [ 0.130880] Detected VIPT I-cache on CPU94 [ 0.130921] GICv3: CPU94: found redistributor 3d0200 region 94:0x00002000aa680000 [ 0.130943] GICv3: CPU94: using allocated LPI pending table @0x0000002080ed0000 [ 0.131038] CPU94: Booted secondary processor 0x00003d0200 [0x481fd010] [ 0.131946] Detected VIPT I-cache on CPU95 [ 0.131987] GICv3: CPU95: found redistributor 3d0300 region 95:0x00002000aa6c0000 [ 0.132009] GICv3: CPU95: using allocated LPI pending table @0x0000002080ee0000 [ 0.132103] CPU95: Booted secondary processor 0x00003d0300 [0x481fd010] [ 0.132838] smp: Brought up 4 nodes, 96 CPUs [ 0.133445] SMP: Total of 96 processors activated. [ 0.133449] CPU features: detected: Privileged Access Never [ 0.133451] CPU features: detected: LSE atomic instructions [ 0.133453] CPU features: detected: Common not Private translations [ 0.133455] CPU features: detected: Data cache clean to Point of Persistence [ 0.133458] CPU features: detected: RAS Extension Support [ 0.133460] CPU features: detected: Data cache clean to the PoU not required for I/D coherence [ 0.133463] CPU features: detected: CRC32 instructions [ 0.133466] CPU features: detected: ARM64 MPAM Extension Support [ 0.133468] CPU features: detected: Taishan IDC coherence workaround [ 0.183674] CPU: All CPU(s) started at EL2 [ 0.226563] devtmpfs: initialized [ 0.232255] KASLR disabled due to lack of seed [ 0.232421] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.232516] futex hash table entries: 32768 (order: 9, 2097152 bytes, vmalloc) [ 0.232897] pinctrl core: initialized pinctrl subsystem [ 0.233100] SMBIOS 3.2.0 present. [ 0.233106] DMI: Huawei TaiShan 200 (Model 2280)/BC82AMDDA, BIOS 1.70 01/07/2021 [ 0.233435] NET: Registered protocol family 16 [ 0.235037] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations [ 0.235593] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.236143] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.236157] audit: initializing netlink subsys (disabled) [ 0.236226] audit: type=2000 audit(0.228:1): state=initialized audit_enabled=0 res=1 [ 0.236363] thermal_sys: Registered thermal governor 'fair_share' [ 0.236365] thermal_sys: Registered thermal governor 'step_wise' [ 0.236369] thermal_sys: Registered thermal governor 'user_space' [ 0.236483] cpuidle: using governor menu [ 0.236592] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers. [ 0.237433] ASID allocator initialised with 65536 entries [ 0.237438] HugeTLB: can optimize 4095 vmemmap pages for hugepages-1048576kB [ 0.237442] HugeTLB: can optimize 127 vmemmap pages for hugepages-32768kB [ 0.237445] HugeTLB: can optimize 7 vmemmap pages for hugepages-2048kB [ 0.237448] HugeTLB: can optimize 0 vmemmap pages for hugepages-64kB [ 0.237474] ACPI: bus type PCI registered [ 0.237477] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.237546] Serial: AMBA PL011 UART driver [ 0.242621] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.242626] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages [ 0.242629] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.242632] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages [ 0.312704] ACPI: Added _OSI(Module Device) [ 0.312709] ACPI: Added _OSI(Processor Device) [ 0.312712] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.312714] ACPI: Added _OSI(Processor Aggregator Device) [ 0.312717] ACPI: Added _OSI(Linux-Dell-Video) [ 0.312720] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio) [ 0.312722] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics) [ 0.316663] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.319198] ACPI: Interpreter enabled [ 0.319203] ACPI: Using GIC for interrupt routing [ 0.319217] ACPI: MCFG table detected, 1 entries [ 0.319222] ACPI: IORT: SMMU-v3[148000000] Mapped to Proximity domain 0 [ 0.319251] ACPI: IORT: SMMU-v3[201000000] Mapped to Proximity domain 0 [ 0.319273] ACPI: IORT: SMMU-v3[100000000] Mapped to Proximity domain 0 [ 0.319289] ACPI: IORT: SMMU-v3[140000000] Mapped to Proximity domain 0 [ 0.319304] ACPI: IORT: SMMU-v3[200148000000] Mapped to Proximity domain 2 [ 0.319321] ACPI: IORT: SMMU-v3[200201000000] Mapped to Proximity domain 2 [ 0.319335] ACPI: IORT: SMMU-v3[200100000000] Mapped to Proximity domain 2 [ 0.319355] ACPI: IORT: SMMU-v3[200140000000] Mapped to Proximity domain 2 [ 0.319658] HEST: Table parsing has been initialized. [ 0.344432] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-3f]) [ 0.344441] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.344519] acpi PNP0A08:00: _OSC: platform does not support [SHPCHotplug LTR DPC] [ 0.344590] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] [ 0.345087] acpi PNP0A08:00: ECAM area [mem 0xd0000000-0xd3ffffff] reserved by PNP0C02:00 [ 0.345096] acpi PNP0A08:00: ECAM at [mem 0xd0000000-0xd3ffffff] for [bus 00-3f] [ 0.345117] Remapped I/O 0x00000000f7ff0000 to [io 0x0000-0xffff window] [ 0.345167] PCI host bridge to bus 0000:00 [ 0.345171] pci_bus 0000:00: root bus resource [mem 0x80000000000-0x82fffffffff pref window] [ 0.345175] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xf7feffff window] [ 0.345178] pci_bus 0000:00: root bus resource [io 0x0000-0xffff window] [ 0.345181] pci_bus 0000:00: root bus resource [bus 00-3f] [ 0.345214] pci 0000:00:00.0: [19e5:a120] type 01 class 0x060400 [ 0.345268] pci 0000:00:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.345331] pci 0000:00:04.0: [19e5:a120] type 01 class 0x060400 [ 0.345379] pci 0000:00:04.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.345430] pci 0000:00:08.0: [19e5:a120] type 01 class 0x060400 [ 0.345473] pci 0000:00:08.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.345521] pci 0000:00:0c.0: [19e5:a120] type 01 class 0x060400 [ 0.345565] pci 0000:00:0c.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.345615] pci 0000:00:10.0: [19e5:a120] type 01 class 0x060400 [ 0.345659] pci 0000:00:10.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.345709] pci 0000:00:11.0: [19e5:a120] type 01 class 0x060400 [ 0.345754] pci 0000:00:11.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.345801] pci 0000:00:12.0: [19e5:a120] type 01 class 0x060400 [ 0.345848] pci 0000:00:12.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.345922] pci 0000:01:00.0: [8086:10fb] type 00 class 0x020000 [ 0.345936] pci 0000:01:00.0: reg 0x10: [mem 0x80000400000-0x800007fffff 64bit pref] [ 0.345943] pci 0000:01:00.0: reg 0x18: [io 0x1020-0x103f] [ 0.345957] pci 0000:01:00.0: reg 0x20: [mem 0x80000c04000-0x80000c07fff 64bit pref] [ 0.345964] pci 0000:01:00.0: reg 0x30: [mem 0xe6400000-0xe67fffff pref] [ 0.346037] pci 0000:01:00.0: PME# supported from D0 D3hot [ 0.346060] pci 0000:01:00.0: reg 0x184: [mem 0x80000b04000-0x80000b07fff 64bit pref] [ 0.346064] pci 0000:01:00.0: VF(n) BAR0 space: [mem 0x80000b04000-0x80000c03fff 64bit pref] (contains BAR0 for 64 VFs) [ 0.346079] pci 0000:01:00.0: reg 0x190: [mem 0x80000a04000-0x80000a07fff 64bit pref] [ 0.346082] pci 0000:01:00.0: VF(n) BAR3 space: [mem 0x80000a04000-0x80000b03fff 64bit pref] (contains BAR3 for 64 VFs) [ 0.346293] pci 0000:01:00.1: [8086:10fb] type 00 class 0x020000 [ 0.346307] pci 0000:01:00.1: reg 0x10: [mem 0x80000000000-0x800003fffff 64bit pref] [ 0.346314] pci 0000:01:00.1: reg 0x18: [io 0x1000-0x101f] [ 0.346328] pci 0000:01:00.1: reg 0x20: [mem 0x80000a00000-0x80000a03fff 64bit pref] [ 0.346335] pci 0000:01:00.1: reg 0x30: [mem 0xe6000000-0xe63fffff pref] [ 0.346404] pci 0000:01:00.1: PME# supported from D0 D3hot [ 0.346425] pci 0000:01:00.1: reg 0x184: [mem 0x80000900000-0x80000903fff 64bit pref] [ 0.346429] pci 0000:01:00.1: VF(n) BAR0 space: [mem 0x80000900000-0x800009fffff 64bit pref] (contains BAR0 for 64 VFs) [ 0.346443] pci 0000:01:00.1: reg 0x190: [mem 0x80000800000-0x80000803fff 64bit pref] [ 0.346446] pci 0000:01:00.1: VF(n) BAR3 space: [mem 0x80000800000-0x800008fffff 64bit pref] (contains BAR3 for 64 VFs) [ 0.346667] pci 0000:03:00.0: [1000:0016] type 00 class 0x010700 [ 0.346681] pci 0000:03:00.0: reg 0x10: [mem 0x80000e00000-0x80000efffff 64bit pref] [ 0.346691] pci 0000:03:00.0: reg 0x18: [mem 0x80000d00000-0x80000dfffff 64bit pref] [ 0.346699] pci 0000:03:00.0: reg 0x20: [mem 0xe6c00000-0xe6cfffff] [ 0.346706] pci 0000:03:00.0: reg 0x24: [io 0x0000-0x00ff] [ 0.346714] pci 0000:03:00.0: reg 0x30: [mem 0xe6b00000-0xe6bfffff pref] [ 0.346778] pci 0000:03:00.0: supports D1 D2 [ 0.346892] pci 0000:05:00.0: [19e5:1710] type 00 class 0x118000 [ 0.346912] pci 0000:05:00.0: reg 0x10: [mem 0xe0000000-0xe3ffffff pref] [ 0.346922] pci 0000:05:00.0: reg 0x14: [mem 0xe6a00000-0xe6afffff] [ 0.347067] pci 0000:05:00.0: supports D1 [ 0.347070] pci 0000:05:00.0: PME# supported from D0 D1 D3hot [ 0.347189] pci 0000:06:00.0: [19e5:1711] type 00 class 0x030000 [ 0.347214] pci 0000:06:00.0: reg 0x10: [mem 0xe4000000-0xe5ffffff pref] [ 0.347227] pci 0000:06:00.0: reg 0x14: [mem 0xe6800000-0xe69fffff] [ 0.347348] pci 0000:06:00.0: BAR 0: assigned to efifb [ 0.347409] pci 0000:06:00.0: supports D1 [ 0.347411] pci 0000:06:00.0: PME# supported from D0 D1 D3hot [ 0.347524] pci_bus 0000:00: on NUMA node 0 [ 0.347526] pci 0000:00:00.0: PCI bridge to [bus 01] [ 0.347530] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff] [ 0.347533] pci 0000:00:00.0: bridge window [mem 0xe6000000-0xe67fffff] [ 0.347537] pci 0000:00:00.0: bridge window [mem 0x80000000000-0x80000cfffff 64bit pref] [ 0.347541] pci 0000:00:04.0: PCI bridge to [bus 02] [ 0.347545] pci 0000:00:08.0: PCI bridge to [bus 03] [ 0.347548] pci 0000:00:08.0: bridge window [io 0x0000-0x0fff] [ 0.347551] pci 0000:00:08.0: bridge window [mem 0xe6b00000-0xe6cfffff] [ 0.347554] pci 0000:00:08.0: bridge window [mem 0x80000d00000-0x80000efffff 64bit pref] [ 0.347558] pci 0000:00:0c.0: PCI bridge to [bus 04] [ 0.347562] pci 0000:00:10.0: PCI bridge to [bus 05] [ 0.347565] pci 0000:00:10.0: bridge window [mem 0xe6a00000-0xe6afffff] [ 0.347569] pci 0000:00:10.0: bridge window [mem 0xe0000000-0xe3ffffff 64bit pref] [ 0.347572] pci 0000:00:11.0: PCI bridge to [bus 06] [ 0.347575] pci 0000:00:11.0: bridge window [mem 0xe6800000-0xe69fffff] [ 0.347579] pci 0000:00:11.0: bridge window [mem 0xe4000000-0xe5ffffff 64bit pref] [ 0.347582] pci 0000:00:12.0: PCI bridge to [bus 07] [ 0.347593] pci 0000:00:00.0: PCI bridge to [bus 01] [ 0.347596] pci 0000:00:00.0: bridge window [io 0x1000-0x1fff] [ 0.347599] pci 0000:00:00.0: bridge window [mem 0xe6000000-0xe67fffff] [ 0.347602] pci 0000:00:00.0: bridge window [mem 0x80000000000-0x80000cfffff 64bit pref] [ 0.347606] pci 0000:00:04.0: PCI bridge to [bus 02] [ 0.347610] pci 0000:00:08.0: PCI bridge to [bus 03] [ 0.347613] pci 0000:00:08.0: bridge window [io 0x0000-0x0fff] [ 0.347616] pci 0000:00:08.0: bridge window [mem 0xe6b00000-0xe6cfffff] [ 0.347620] pci 0000:00:08.0: bridge window [mem 0x80000d00000-0x80000efffff 64bit pref] [ 0.347623] pci 0000:00:0c.0: PCI bridge to [bus 04] [ 0.347628] pci 0000:00:10.0: PCI bridge to [bus 05] [ 0.347631] pci 0000:00:10.0: bridge window [mem 0xe6a00000-0xe6afffff] [ 0.347634] pci 0000:00:10.0: bridge window [mem 0xe0000000-0xe3ffffff 64bit pref] [ 0.347638] pci 0000:00:11.0: PCI bridge to [bus 06] [ 0.347641] pci 0000:00:11.0: bridge window [mem 0xe6800000-0xe69fffff] [ 0.347644] pci 0000:00:11.0: bridge window [mem 0xe4000000-0xe5ffffff 64bit pref] [ 0.347648] pci 0000:00:12.0: PCI bridge to [bus 07] [ 0.347652] pci_bus 0000:00: resource 4 [mem 0x80000000000-0x82fffffffff pref window] [ 0.347655] pci_bus 0000:00: resource 5 [mem 0xe0000000-0xf7feffff window] [ 0.347658] pci_bus 0000:00: resource 6 [io 0x0000-0xffff window] [ 0.347661] pci_bus 0000:01: resource 0 [io 0x1000-0x1fff] [ 0.347663] pci_bus 0000:01: resource 1 [mem 0xe6000000-0xe67fffff] [ 0.347666] pci_bus 0000:01: resource 2 [mem 0x80000000000-0x80000cfffff 64bit pref] [ 0.347669] pci_bus 0000:03: resource 0 [io 0x0000-0x0fff] [ 0.347671] pci_bus 0000:03: resource 1 [mem 0xe6b00000-0xe6cfffff] [ 0.347674] pci_bus 0000:03: resource 2 [mem 0x80000d00000-0x80000efffff 64bit pref] [ 0.347677] pci_bus 0000:05: resource 1 [mem 0xe6a00000-0xe6afffff] [ 0.347679] pci_bus 0000:05: resource 2 [mem 0xe0000000-0xe3ffffff 64bit pref] [ 0.347682] pci_bus 0000:06: resource 1 [mem 0xe6800000-0xe69fffff] [ 0.347685] pci_bus 0000:06: resource 2 [mem 0xe4000000-0xe5ffffff 64bit pref] [ 0.347733] ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 7b]) [ 0.347738] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.347816] acpi PNP0A08:01: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.347881] acpi PNP0A08:01: _OSC: OS now controls [PCIeCapability] [ 0.348379] acpi PNP0A08:01: ECAM area [mem 0xd7b00000-0xd7bfffff] reserved by PNP0C02:00 [ 0.348394] acpi PNP0A08:01: ECAM at [mem 0xd7b00000-0xd7bfffff] for [bus 7b] [ 0.348437] PCI host bridge to bus 0000:7b [ 0.348441] pci_bus 0000:7b: root bus resource [mem 0x148800000-0x148ffffff pref window] [ 0.348444] pci_bus 0000:7b: root bus resource [bus 7b] [ 0.348452] pci 0000:7b:00.0: [19e5:a122] type 00 class 0x088000 [ 0.348459] pci 0000:7b:00.0: reg 0x18: [mem 0x00000000-0x00003fff 64bit pref] [ 0.348513] pci_bus 0000:7b: on NUMA node 0 [ 0.348517] pci 0000:7b:00.0: BAR 2: assigned [mem 0x148800000-0x148803fff 64bit pref] [ 0.348522] pci_bus 0000:7b: resource 4 [mem 0x148800000-0x148ffffff pref window] [ 0.348562] ACPI: PCI Root Bridge [PCI2] (domain 0000 [bus 7a]) [ 0.348567] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.348639] acpi PNP0A08:02: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.348704] acpi PNP0A08:02: _OSC: OS now controls [PCIeCapability] [ 0.349191] acpi PNP0A08:02: ECAM area [mem 0xd7a00000-0xd7afffff] reserved by PNP0C02:00 [ 0.349205] acpi PNP0A08:02: ECAM at [mem 0xd7a00000-0xd7afffff] for [bus 7a] [ 0.349253] PCI host bridge to bus 0000:7a [ 0.349257] pci_bus 0000:7a: root bus resource [mem 0x20c000000-0x20c1fffff pref window] [ 0.349260] pci_bus 0000:7a: root bus resource [bus 7a] [ 0.349267] pci 0000:7a:00.0: [19e5:a23b] type 00 class 0x0c0310 [ 0.349273] pci 0000:7a:00.0: reg 0x10: [mem 0x20c100000-0x20c100fff 64bit pref] [ 0.349320] pci 0000:7a:01.0: [19e5:a239] type 00 class 0x0c0320 [ 0.349326] pci 0000:7a:01.0: reg 0x10: [mem 0x20c101000-0x20c101fff 64bit pref] [ 0.349372] pci 0000:7a:02.0: [19e5:a238] type 00 class 0x0c0330 [ 0.349377] pci 0000:7a:02.0: reg 0x10: [mem 0x20c000000-0x20c0fffff 64bit pref] [ 0.349427] pci_bus 0000:7a: on NUMA node 0 [ 0.349431] pci 0000:7a:02.0: BAR 0: assigned [mem 0x20c000000-0x20c0fffff 64bit pref] [ 0.349436] pci 0000:7a:00.0: BAR 0: assigned [mem 0x20c100000-0x20c100fff 64bit pref] [ 0.349440] pci 0000:7a:01.0: BAR 0: assigned [mem 0x20c101000-0x20c101fff 64bit pref] [ 0.349445] pci_bus 0000:7a: resource 4 [mem 0x20c000000-0x20c1fffff pref window] [ 0.349484] ACPI: PCI Root Bridge [PCI3] (domain 0000 [bus 78-79]) [ 0.349489] acpi PNP0A08:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.349559] acpi PNP0A08:03: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.349623] acpi PNP0A08:03: _OSC: OS now controls [PCIeCapability] [ 0.350107] acpi PNP0A08:03: ECAM area [mem 0xd7800000-0xd79fffff] reserved by PNP0C02:00 [ 0.350114] acpi PNP0A08:03: ECAM at [mem 0xd7800000-0xd79fffff] for [bus 78-79] [ 0.350175] PCI host bridge to bus 0000:78 [ 0.350178] pci_bus 0000:78: root bus resource [mem 0x208000000-0x208bfffff pref window] [ 0.350182] pci_bus 0000:78: root bus resource [bus 78-79] [ 0.350192] pci_bus 0000:78: on NUMA node 0 [ 0.350194] pci_bus 0000:78: resource 4 [mem 0x208000000-0x208bfffff pref window] [ 0.350231] ACPI: PCI Root Bridge [PCI4] (domain 0000 [bus 7c-7d]) [ 0.350235] acpi PNP0A08:04: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.350305] acpi PNP0A08:04: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.350368] acpi PNP0A08:04: _OSC: OS now controls [PCIeCapability] [ 0.350850] acpi PNP0A08:04: ECAM area [mem 0xd7c00000-0xd7dfffff] reserved by PNP0C02:00 [ 0.350855] acpi PNP0A08:04: ECAM at [mem 0xd7c00000-0xd7dfffff] for [bus 7c-7d] [ 0.350902] PCI host bridge to bus 0000:7c [ 0.350905] pci_bus 0000:7c: root bus resource [mem 0x120000000-0x13fffffff pref window] [ 0.350909] pci_bus 0000:7c: root bus resource [bus 7c-7d] [ 0.350916] pci 0000:7c:00.0: [19e5:a121] type 01 class 0x060400 [ 0.350925] pci 0000:7c:00.0: enabling Extended Tags [ 0.350939] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0x7c00 [ 0.350942] ACPI: IORT: [Firmware Bug]: applying workaround. [ 0.350995] pci 0000:7d:00.0: [19e5:a222] type 00 class 0x020000 [ 0.351001] pci 0000:7d:00.0: reg 0x10: [mem 0x1221f0000-0x1221fffff 64bit pref] [ 0.351006] pci 0000:7d:00.0: reg 0x18: [mem 0x121f00000-0x121ffffff 64bit pref] [ 0.351027] pci 0000:7d:00.0: reg 0x224: [mem 0x122180000-0x12218ffff 64bit pref] [ 0.351030] pci 0000:7d:00.0: VF(n) BAR0 space: [mem 0x122180000-0x1221effff 64bit pref] (contains BAR0 for 7 VFs) [ 0.351035] pci 0000:7d:00.0: reg 0x22c: [mem 0x121800000-0x1218fffff 64bit pref] [ 0.351039] pci 0000:7d:00.0: VF(n) BAR2 space: [mem 0x121800000-0x121efffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.351084] pci 0000:7d:00.1: [19e5:a221] type 00 class 0x020000 [ 0.351089] pci 0000:7d:00.1: reg 0x10: [mem 0x122170000-0x12217ffff 64bit pref] [ 0.351094] pci 0000:7d:00.1: reg 0x18: [mem 0x121700000-0x1217fffff 64bit pref] [ 0.351114] pci 0000:7d:00.1: reg 0x224: [mem 0x122100000-0x12210ffff 64bit pref] [ 0.351117] pci 0000:7d:00.1: VF(n) BAR0 space: [mem 0x122100000-0x12216ffff 64bit pref] (contains BAR0 for 7 VFs) [ 0.351122] pci 0000:7d:00.1: reg 0x22c: [mem 0x121000000-0x1210fffff 64bit pref] [ 0.351125] pci 0000:7d:00.1: VF(n) BAR2 space: [mem 0x121000000-0x1216fffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.351174] pci 0000:7d:00.2: [19e5:a222] type 00 class 0x020000 [ 0.351180] pci 0000:7d:00.2: reg 0x10: [mem 0x1220f0000-0x1220fffff 64bit pref] [ 0.351185] pci 0000:7d:00.2: reg 0x18: [mem 0x120f00000-0x120ffffff 64bit pref] [ 0.351204] pci 0000:7d:00.2: reg 0x224: [mem 0x122080000-0x12208ffff 64bit pref] [ 0.351207] pci 0000:7d:00.2: VF(n) BAR0 space: [mem 0x122080000-0x1220effff 64bit pref] (contains BAR0 for 7 VFs) [ 0.351213] pci 0000:7d:00.2: reg 0x22c: [mem 0x120800000-0x1208fffff 64bit pref] [ 0.351216] pci 0000:7d:00.2: VF(n) BAR2 space: [mem 0x120800000-0x120efffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.351258] pci 0000:7d:00.3: [19e5:a221] type 00 class 0x020000 [ 0.351263] pci 0000:7d:00.3: reg 0x10: [mem 0x122070000-0x12207ffff 64bit pref] [ 0.351268] pci 0000:7d:00.3: reg 0x18: [mem 0x120700000-0x1207fffff 64bit pref] [ 0.351287] pci 0000:7d:00.3: reg 0x224: [mem 0x122000000-0x12200ffff 64bit pref] [ 0.351290] pci 0000:7d:00.3: VF(n) BAR0 space: [mem 0x122000000-0x12206ffff 64bit pref] (contains BAR0 for 7 VFs) [ 0.351295] pci 0000:7d:00.3: reg 0x22c: [mem 0x120000000-0x1200fffff 64bit pref] [ 0.351298] pci 0000:7d:00.3: VF(n) BAR2 space: [mem 0x120000000-0x1206fffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.351342] pci_bus 0000:7c: on NUMA node 0 [ 0.351349] pci 0000:7c:00.0: bridge window [mem 0x00100000-0x005fffff 64bit pref] to [bus 7d] add_size 1d00000 add_align 100000 [ 0.351354] pci 0000:7c:00.0: BAR 15: assigned [mem 0x120000000-0x1221fffff 64bit pref] [ 0.351362] pci 0000:7d:00.0: BAR 2: assigned [mem 0x120000000-0x1200fffff 64bit pref] [ 0.351366] pci 0000:7d:00.0: BAR 9: assigned [mem 0x120100000-0x1207fffff 64bit pref] [ 0.351370] pci 0000:7d:00.1: BAR 2: assigned [mem 0x120800000-0x1208fffff 64bit pref] [ 0.351374] pci 0000:7d:00.1: BAR 9: assigned [mem 0x120900000-0x120ffffff 64bit pref] [ 0.351378] pci 0000:7d:00.2: BAR 2: assigned [mem 0x121000000-0x1210fffff 64bit pref] [ 0.351382] pci 0000:7d:00.2: BAR 9: assigned [mem 0x121100000-0x1217fffff 64bit pref] [ 0.351386] pci 0000:7d:00.3: BAR 2: assigned [mem 0x121800000-0x1218fffff 64bit pref] [ 0.351390] pci 0000:7d:00.3: BAR 9: assigned [mem 0x121900000-0x121ffffff 64bit pref] [ 0.351393] pci 0000:7d:00.0: BAR 0: assigned [mem 0x122000000-0x12200ffff 64bit pref] [ 0.351397] pci 0000:7d:00.0: BAR 7: assigned [mem 0x122010000-0x12207ffff 64bit pref] [ 0.351401] pci 0000:7d:00.1: BAR 0: assigned [mem 0x122080000-0x12208ffff 64bit pref] [ 0.351405] pci 0000:7d:00.1: BAR 7: assigned [mem 0x122090000-0x1220fffff 64bit pref] [ 0.351408] pci 0000:7d:00.2: BAR 0: assigned [mem 0x122100000-0x12210ffff 64bit pref] [ 0.351413] pci 0000:7d:00.2: BAR 7: assigned [mem 0x122110000-0x12217ffff 64bit pref] [ 0.351416] pci 0000:7d:00.3: BAR 0: assigned [mem 0x122180000-0x12218ffff 64bit pref] [ 0.351420] pci 0000:7d:00.3: BAR 7: assigned [mem 0x122190000-0x1221fffff 64bit pref] [ 0.351426] pci 0000:7c:00.0: PCI bridge to [bus 7d] [ 0.351429] pci 0000:7c:00.0: bridge window [mem 0x120000000-0x1221fffff 64bit pref] [ 0.351433] pci_bus 0000:7c: resource 4 [mem 0x120000000-0x13fffffff pref window] [ 0.351436] pci_bus 0000:7d: resource 2 [mem 0x120000000-0x1221fffff 64bit pref] [ 0.351480] ACPI: PCI Root Bridge [PCI5] (domain 0000 [bus 74-75]) [ 0.351485] acpi PNP0A08:05: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.351556] acpi PNP0A08:05: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.351620] acpi PNP0A08:05: _OSC: OS now controls [PCIeCapability] [ 0.352110] acpi PNP0A08:05: ECAM area [mem 0xd7400000-0xd75fffff] reserved by PNP0C02:00 [ 0.352116] acpi PNP0A08:05: ECAM at [mem 0xd7400000-0xd75fffff] for [bus 74-75] [ 0.352186] PCI host bridge to bus 0000:74 [ 0.352190] pci_bus 0000:74: root bus resource [mem 0x141000000-0x141ffffff pref window] [ 0.352193] pci_bus 0000:74: root bus resource [mem 0x144000000-0x145ffffff pref window] [ 0.352196] pci_bus 0000:74: root bus resource [mem 0xa2000000-0xa2ffffff window] [ 0.352199] pci_bus 0000:74: root bus resource [bus 74-75] [ 0.352207] pci 0000:74:01.0: [19e5:a121] type 01 class 0x060400 [ 0.352215] pci 0000:74:01.0: enabling Extended Tags [ 0.352281] pci 0000:74:02.0: [19e5:a230] type 00 class 0x010700 [ 0.352289] pci 0000:74:02.0: reg 0x24: [mem 0xa2008000-0xa200ffff] [ 0.352367] pci 0000:74:03.0: [19e5:a235] type 00 class 0x010601 [ 0.352378] pci 0000:74:03.0: reg 0x24: [mem 0xa2010000-0xa2010fff] [ 0.352426] pci 0000:74:04.0: [19e5:a230] type 00 class 0x010700 [ 0.352434] pci 0000:74:04.0: reg 0x24: [mem 0xa2000000-0xa2007fff] [ 0.352541] pci_bus 0000:76: busn_res: can not insert [bus 76] under [bus 74-75] (conflicts with (null) [bus 74-75]) [ 0.352552] pci_bus 0000:74: on NUMA node 0 [ 0.352556] pci 0000:74:02.0: BAR 5: assigned [mem 0xa2000000-0xa2007fff] [ 0.352560] pci 0000:74:04.0: BAR 5: assigned [mem 0xa2008000-0xa200ffff] [ 0.352564] pci 0000:74:03.0: BAR 5: assigned [mem 0xa2010000-0xa2010fff] [ 0.352567] pci 0000:74:01.0: PCI bridge to [bus 76] [ 0.352572] pci_bus 0000:74: resource 4 [mem 0x141000000-0x141ffffff pref window] [ 0.352575] pci_bus 0000:74: resource 5 [mem 0x144000000-0x145ffffff pref window] [ 0.352578] pci_bus 0000:74: resource 6 [mem 0xa2000000-0xa2ffffff window] [ 0.352641] ACPI: PCI Root Bridge [PCI6] (domain 0000 [bus 80-9f]) [ 0.352646] acpi PNP0A08:06: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.352718] acpi PNP0A08:06: _OSC: platform does not support [SHPCHotplug LTR DPC] [ 0.352782] acpi PNP0A08:06: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] [ 0.353267] acpi PNP0A08:06: ECAM area [mem 0xd8000000-0xd9ffffff] reserved by PNP0C02:00 [ 0.353274] acpi PNP0A08:06: ECAM at [mem 0xd8000000-0xd9ffffff] for [bus 80-9f] [ 0.353300] Remapped I/O 0x00000000cfff0000 to [io 0x10000-0x1ffff window] [ 0.353360] PCI host bridge to bus 0000:80 [ 0.353364] pci_bus 0000:80: root bus resource [mem 0x280000000000-0x282fffffffff pref window] [ 0.353367] pci_bus 0000:80: root bus resource [mem 0xb0000000-0xcffeffff window] [ 0.353371] pci_bus 0000:80: root bus resource [io 0x10000-0x1ffff window] (bus address [0x0000-0xffff]) [ 0.353374] pci_bus 0000:80: root bus resource [bus 80-9f] [ 0.353402] pci 0000:80:00.0: [19e5:a120] type 01 class 0x060400 [ 0.353465] pci 0000:80:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.353533] pci 0000:80:08.0: [19e5:a120] type 01 class 0x060400 [ 0.353585] pci 0000:80:08.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.353643] pci 0000:80:0a.0: [19e5:a120] type 01 class 0x060400 [ 0.353695] pci 0000:80:0a.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.353753] pci 0000:80:0c.0: [19e5:a120] type 01 class 0x060400 [ 0.353808] pci 0000:80:0c.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.353866] pci 0000:80:0e.0: [19e5:a120] type 01 class 0x060400 [ 0.353920] pci 0000:80:0e.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.353975] pci 0000:80:10.0: [19e5:a120] type 01 class 0x060400 [ 0.354028] pci 0000:80:10.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.354279] pci 0000:81:00.0: [15b3:1017] type 00 class 0x020700 [ 0.354474] pci 0000:81:00.0: reg 0x10: [mem 0x280010000000-0x28001fffffff 64bit pref] [ 0.354869] pci 0000:81:00.0: reg 0x30: [mem 0xb0500000-0xb05fffff pref] [ 0.355903] pci 0000:81:00.0: PME# supported from D3cold [ 0.356246] pci 0000:81:00.0: reg 0x1a4: [mem 0x280020400000-0x2800205fffff 64bit pref] [ 0.356249] pci 0000:81:00.0: VF(n) BAR0 space: [mem 0x280020400000-0x2800207fffff 64bit pref] (contains BAR0 for 2 VFs) [ 0.357449] pci 0000:81:00.1: [15b3:1017] type 00 class 0x020700 [ 0.357642] pci 0000:81:00.1: reg 0x10: [mem 0x280000000000-0x28000fffffff 64bit pref] [ 0.358039] pci 0000:81:00.1: reg 0x30: [mem 0xb0400000-0xb04fffff pref] [ 0.359022] pci 0000:81:00.1: PME# supported from D3cold [ 0.359348] pci 0000:81:00.1: reg 0x1a4: [mem 0x280020000000-0x2800201fffff 64bit pref] [ 0.359351] pci 0000:81:00.1: VF(n) BAR0 space: [mem 0x280020000000-0x2800203fffff 64bit pref] (contains BAR0 for 2 VFs) [ 0.360383] pci 0000:82:00.0: [19e5:3754] type 00 class 0x010802 [ 0.360394] pci 0000:82:00.0: reg 0x10: [mem 0xb0340000-0xb034ffff 64bit] [ 0.360418] pci 0000:82:00.0: reg 0x24: [mem 0xb0320000-0xb033ffff] [ 0.360424] pci 0000:82:00.0: reg 0x30: [mem 0xb0300000-0xb031ffff pref] [ 0.360581] pci 0000:83:00.0: [19e5:3754] type 00 class 0x010802 [ 0.360595] pci 0000:83:00.0: reg 0x10: [mem 0xb0240000-0xb024ffff 64bit] [ 0.360620] pci 0000:83:00.0: reg 0x24: [mem 0xb0220000-0xb023ffff] [ 0.360626] pci 0000:83:00.0: reg 0x30: [mem 0xb0200000-0xb021ffff pref] [ 0.360783] pci 0000:84:00.0: [19e5:3754] type 00 class 0x010802 [ 0.360796] pci 0000:84:00.0: reg 0x10: [mem 0xb0140000-0xb014ffff 64bit] [ 0.360821] pci 0000:84:00.0: reg 0x24: [mem 0xb0120000-0xb013ffff] [ 0.360827] pci 0000:84:00.0: reg 0x30: [mem 0xb0100000-0xb011ffff pref] [ 0.360986] pci 0000:85:00.0: [19e5:3754] type 00 class 0x010802 [ 0.360997] pci 0000:85:00.0: reg 0x10: [mem 0xb0040000-0xb004ffff 64bit] [ 0.361022] pci 0000:85:00.0: reg 0x24: [mem 0xb0020000-0xb003ffff] [ 0.361029] pci 0000:85:00.0: reg 0x30: [mem 0xb0000000-0xb001ffff pref] [ 0.361185] pci_bus 0000:80: on NUMA node 2 [ 0.361187] pci 0000:80:00.0: PCI bridge to [bus 81] [ 0.361191] pci 0000:80:00.0: bridge window [mem 0xb0400000-0xb05fffff] [ 0.361195] pci 0000:80:00.0: bridge window [mem 0x280000000000-0x2800207fffff 64bit pref] [ 0.361199] pci 0000:80:08.0: PCI bridge to [bus 82] [ 0.361203] pci 0000:80:08.0: bridge window [mem 0xb0300000-0xb03fffff] [ 0.361207] pci 0000:80:0a.0: PCI bridge to [bus 83] [ 0.361211] pci 0000:80:0a.0: bridge window [mem 0xb0200000-0xb02fffff] [ 0.361215] pci 0000:80:0c.0: PCI bridge to [bus 84] [ 0.361219] pci 0000:80:0c.0: bridge window [mem 0xb0100000-0xb01fffff] [ 0.361222] pci 0000:80:0e.0: PCI bridge to [bus 85] [ 0.361226] pci 0000:80:0e.0: bridge window [mem 0xb0000000-0xb00fffff] [ 0.361230] pci 0000:80:10.0: PCI bridge to [bus 86] [ 0.361238] pci 0000:80:08.0: bridge window [io 0x1000-0x0fff] to [bus 82] add_size 1000 [ 0.361242] pci 0000:80:08.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 82] add_size 200000 add_align 100000 [ 0.361246] pci 0000:80:0a.0: bridge window [io 0x1000-0x0fff] to [bus 83] add_size 1000 [ 0.361252] pci 0000:80:0a.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 83] add_size 200000 add_align 100000 [ 0.361255] pci 0000:80:0c.0: bridge window [io 0x1000-0x0fff] to [bus 84] add_size 1000 [ 0.361259] pci 0000:80:0c.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 84] add_size 200000 add_align 100000 [ 0.361262] pci 0000:80:0e.0: bridge window [io 0x1000-0x0fff] to [bus 85] add_size 1000 [ 0.361266] pci 0000:80:0e.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 85] add_size 200000 add_align 100000 [ 0.361272] pci 0000:80:08.0: BAR 15: assigned [mem 0x280020800000-0x2800209fffff 64bit pref] [ 0.361275] pci 0000:80:0a.0: BAR 15: assigned [mem 0x280020a00000-0x280020bfffff 64bit pref] [ 0.361279] pci 0000:80:0c.0: BAR 15: assigned [mem 0x280020c00000-0x280020dfffff 64bit pref] [ 0.361282] pci 0000:80:0e.0: BAR 15: assigned [mem 0x280020e00000-0x280020ffffff 64bit pref] [ 0.361285] pci 0000:80:08.0: BAR 13: assigned [io 0x10000-0x10fff] [ 0.361288] pci 0000:80:0a.0: BAR 13: assigned [io 0x11000-0x11fff] [ 0.361291] pci 0000:80:0c.0: BAR 13: assigned [io 0x12000-0x12fff] [ 0.361293] pci 0000:80:0e.0: BAR 13: assigned [io 0x13000-0x13fff] [ 0.361297] pci 0000:80:00.0: PCI bridge to [bus 81] [ 0.361300] pci 0000:80:00.0: bridge window [mem 0xb0400000-0xb05fffff] [ 0.361304] pci 0000:80:00.0: bridge window [mem 0x280000000000-0x2800207fffff 64bit pref] [ 0.361308] pci 0000:80:08.0: PCI bridge to [bus 82] [ 0.361311] pci 0000:80:08.0: bridge window [io 0x10000-0x10fff] [ 0.361314] pci 0000:80:08.0: bridge window [mem 0xb0300000-0xb03fffff] [ 0.361318] pci 0000:80:08.0: bridge window [mem 0x280020800000-0x2800209fffff 64bit pref] [ 0.361322] pci 0000:80:0a.0: PCI bridge to [bus 83] [ 0.361324] pci 0000:80:0a.0: bridge window [io 0x11000-0x11fff] [ 0.361327] pci 0000:80:0a.0: bridge window [mem 0xb0200000-0xb02fffff] [ 0.361331] pci 0000:80:0a.0: bridge window [mem 0x280020a00000-0x280020bfffff 64bit pref] [ 0.361335] pci 0000:80:0c.0: PCI bridge to [bus 84] [ 0.361337] pci 0000:80:0c.0: bridge window [io 0x12000-0x12fff] [ 0.361341] pci 0000:80:0c.0: bridge window [mem 0xb0100000-0xb01fffff] [ 0.361344] pci 0000:80:0c.0: bridge window [mem 0x280020c00000-0x280020dfffff 64bit pref] [ 0.361348] pci 0000:80:0e.0: PCI bridge to [bus 85] [ 0.361351] pci 0000:80:0e.0: bridge window [io 0x13000-0x13fff] [ 0.361354] pci 0000:80:0e.0: bridge window [mem 0xb0000000-0xb00fffff] [ 0.361358] pci 0000:80:0e.0: bridge window [mem 0x280020e00000-0x280020ffffff 64bit pref] [ 0.361361] pci 0000:80:10.0: PCI bridge to [bus 86] [ 0.361367] pci_bus 0000:80: resource 4 [mem 0x280000000000-0x282fffffffff pref window] [ 0.361370] pci_bus 0000:80: resource 5 [mem 0xb0000000-0xcffeffff window] [ 0.361372] pci_bus 0000:80: resource 6 [io 0x10000-0x1ffff window] [ 0.361375] pci_bus 0000:81: resource 1 [mem 0xb0400000-0xb05fffff] [ 0.361378] pci_bus 0000:81: resource 2 [mem 0x280000000000-0x2800207fffff 64bit pref] [ 0.361381] pci_bus 0000:82: resource 0 [io 0x10000-0x10fff] [ 0.361383] pci_bus 0000:82: resource 1 [mem 0xb0300000-0xb03fffff] [ 0.361386] pci_bus 0000:82: resource 2 [mem 0x280020800000-0x2800209fffff 64bit pref] [ 0.361389] pci_bus 0000:83: resource 0 [io 0x11000-0x11fff] [ 0.361391] pci_bus 0000:83: resource 1 [mem 0xb0200000-0xb02fffff] [ 0.361394] pci_bus 0000:83: resource 2 [mem 0x280020a00000-0x280020bfffff 64bit pref] [ 0.361397] pci_bus 0000:84: resource 0 [io 0x12000-0x12fff] [ 0.361399] pci_bus 0000:84: resource 1 [mem 0xb0100000-0xb01fffff] [ 0.361402] pci_bus 0000:84: resource 2 [mem 0x280020c00000-0x280020dfffff 64bit pref] [ 0.361405] pci_bus 0000:85: resource 0 [io 0x13000-0x13fff] [ 0.361407] pci_bus 0000:85: resource 1 [mem 0xb0000000-0xb00fffff] [ 0.361410] pci_bus 0000:85: resource 2 [mem 0x280020e00000-0x280020ffffff 64bit pref] [ 0.361465] ACPI: PCI Root Bridge [PCI7] (domain 0000 [bus bb]) [ 0.361471] acpi PNP0A08:07: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.361544] acpi PNP0A08:07: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.361609] acpi PNP0A08:07: _OSC: OS now controls [PCIeCapability] [ 0.362099] acpi PNP0A08:07: ECAM area [mem 0xdbb00000-0xdbbfffff] reserved by PNP0C02:00 [ 0.362114] acpi PNP0A08:07: ECAM at [mem 0xdbb00000-0xdbbfffff] for [bus bb] [ 0.362180] PCI host bridge to bus 0000:bb [ 0.362184] pci_bus 0000:bb: root bus resource [mem 0x200148800000-0x200148ffffff pref window] [ 0.362187] pci_bus 0000:bb: root bus resource [bus bb] [ 0.362196] pci 0000:bb:00.0: [19e5:a122] type 00 class 0x088000 [ 0.362204] pci 0000:bb:00.0: reg 0x18: [mem 0x00000000-0x00003fff 64bit pref] [ 0.362258] pci_bus 0000:bb: on NUMA node 2 [ 0.362261] pci 0000:bb:00.0: BAR 2: assigned [mem 0x200148800000-0x200148803fff 64bit pref] [ 0.362267] pci_bus 0000:bb: resource 4 [mem 0x200148800000-0x200148ffffff pref window] [ 0.362311] ACPI: PCI Root Bridge [PCI8] (domain 0000 [bus ba]) [ 0.362316] acpi PNP0A08:08: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.362390] acpi PNP0A08:08: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.362454] acpi PNP0A08:08: _OSC: OS now controls [PCIeCapability] [ 0.362943] acpi PNP0A08:08: ECAM area [mem 0xdba00000-0xdbafffff] reserved by PNP0C02:00 [ 0.362957] acpi PNP0A08:08: ECAM at [mem 0xdba00000-0xdbafffff] for [bus ba] [ 0.363018] PCI host bridge to bus 0000:ba [ 0.363022] pci_bus 0000:ba: root bus resource [mem 0x20020c000000-0x20020c1fffff pref window] [ 0.363025] pci_bus 0000:ba: root bus resource [bus ba] [ 0.363033] pci 0000:ba:00.0: [19e5:a23b] type 00 class 0x0c0310 [ 0.363039] pci 0000:ba:00.0: reg 0x10: [mem 0x20020c100000-0x20020c100fff 64bit pref] [ 0.363093] pci 0000:ba:01.0: [19e5:a239] type 00 class 0x0c0320 [ 0.363100] pci 0000:ba:01.0: reg 0x10: [mem 0x20020c101000-0x20020c101fff 64bit pref] [ 0.363157] pci 0000:ba:02.0: [19e5:a238] type 00 class 0x0c0330 [ 0.363164] pci 0000:ba:02.0: reg 0x10: [mem 0x20020c000000-0x20020c0fffff 64bit pref] [ 0.363219] pci_bus 0000:ba: on NUMA node 2 [ 0.363222] pci 0000:ba:02.0: BAR 0: assigned [mem 0x20020c000000-0x20020c0fffff 64bit pref] [ 0.363228] pci 0000:ba:00.0: BAR 0: assigned [mem 0x20020c100000-0x20020c100fff 64bit pref] [ 0.363233] pci 0000:ba:01.0: BAR 0: assigned [mem 0x20020c101000-0x20020c101fff 64bit pref] [ 0.363238] pci_bus 0000:ba: resource 4 [mem 0x20020c000000-0x20020c1fffff pref window] [ 0.363285] ACPI: PCI Root Bridge [PCI9] (domain 0000 [bus b8-b9]) [ 0.363289] acpi PNP0A08:09: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.363360] acpi PNP0A08:09: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.363424] acpi PNP0A08:09: _OSC: OS now controls [PCIeCapability] [ 0.363909] acpi PNP0A08:09: ECAM area [mem 0xdb800000-0xdb9fffff] reserved by PNP0C02:00 [ 0.363915] acpi PNP0A08:09: ECAM at [mem 0xdb800000-0xdb9fffff] for [bus b8-b9] [ 0.363991] PCI host bridge to bus 0000:b8 [ 0.363995] pci_bus 0000:b8: root bus resource [mem 0x200208000000-0x200208bfffff pref window] [ 0.363998] pci_bus 0000:b8: root bus resource [bus b8-b9] [ 0.364011] pci_bus 0000:b8: on NUMA node 2 [ 0.364013] pci_bus 0000:b8: resource 4 [mem 0x200208000000-0x200208bfffff pref window] [ 0.364060] ACPI: PCI Root Bridge [PCIA] (domain 0000 [bus bc-bd]) [ 0.364065] acpi PNP0A08:0a: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.364135] acpi PNP0A08:0a: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.364198] acpi PNP0A08:0a: _OSC: OS now controls [PCIeCapability] [ 0.364688] acpi PNP0A08:0a: ECAM area [mem 0xdbc00000-0xdbdfffff] reserved by PNP0C02:00 [ 0.364695] acpi PNP0A08:0a: ECAM at [mem 0xdbc00000-0xdbdfffff] for [bus bc-bd] [ 0.364763] PCI host bridge to bus 0000:bc [ 0.364767] pci_bus 0000:bc: root bus resource [mem 0x200120000000-0x20013fffffff pref window] [ 0.364770] pci_bus 0000:bc: root bus resource [bus bc-bd] [ 0.364778] pci 0000:bc:00.0: [19e5:a121] type 01 class 0x060400 [ 0.364789] pci 0000:bc:00.0: enabling Extended Tags [ 0.364806] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0xbc00 [ 0.364809] ACPI: IORT: [Firmware Bug]: applying workaround. [ 0.364878] pci 0000:bd:00.0: [19e5:a222] type 00 class 0x020000 [ 0.364884] pci 0000:bd:00.0: reg 0x10: [mem 0x2001221f0000-0x2001221fffff 64bit pref] [ 0.364890] pci 0000:bd:00.0: reg 0x18: [mem 0x200121f00000-0x200121ffffff 64bit pref] [ 0.364917] pci 0000:bd:00.0: reg 0x224: [mem 0x200122180000-0x20012218ffff 64bit pref] [ 0.364920] pci 0000:bd:00.0: VF(n) BAR0 space: [mem 0x200122180000-0x2001221effff 64bit pref] (contains BAR0 for 7 VFs) [ 0.364926] pci 0000:bd:00.0: reg 0x22c: [mem 0x200121800000-0x2001218fffff 64bit pref] [ 0.364929] pci 0000:bd:00.0: VF(n) BAR2 space: [mem 0x200121800000-0x200121efffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.364981] pci 0000:bd:00.1: [19e5:a221] type 00 class 0x020000 [ 0.364987] pci 0000:bd:00.1: reg 0x10: [mem 0x200122170000-0x20012217ffff 64bit pref] [ 0.364993] pci 0000:bd:00.1: reg 0x18: [mem 0x200121700000-0x2001217fffff 64bit pref] [ 0.365019] pci 0000:bd:00.1: reg 0x224: [mem 0x200122100000-0x20012210ffff 64bit pref] [ 0.365022] pci 0000:bd:00.1: VF(n) BAR0 space: [mem 0x200122100000-0x20012216ffff 64bit pref] (contains BAR0 for 7 VFs) [ 0.365028] pci 0000:bd:00.1: reg 0x22c: [mem 0x200121000000-0x2001210fffff 64bit pref] [ 0.365031] pci 0000:bd:00.1: VF(n) BAR2 space: [mem 0x200121000000-0x2001216fffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.365077] pci 0000:bd:00.2: [19e5:a222] type 00 class 0x020000 [ 0.365084] pci 0000:bd:00.2: reg 0x10: [mem 0x2001220f0000-0x2001220fffff 64bit pref] [ 0.365090] pci 0000:bd:00.2: reg 0x18: [mem 0x200120f00000-0x200120ffffff 64bit pref] [ 0.365116] pci 0000:bd:00.2: reg 0x224: [mem 0x200122080000-0x20012208ffff 64bit pref] [ 0.365119] pci 0000:bd:00.2: VF(n) BAR0 space: [mem 0x200122080000-0x2001220effff 64bit pref] (contains BAR0 for 7 VFs) [ 0.365125] pci 0000:bd:00.2: reg 0x22c: [mem 0x200120800000-0x2001208fffff 64bit pref] [ 0.365128] pci 0000:bd:00.2: VF(n) BAR2 space: [mem 0x200120800000-0x200120efffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.365175] pci 0000:bd:00.3: [19e5:a221] type 00 class 0x020000 [ 0.365181] pci 0000:bd:00.3: reg 0x10: [mem 0x200122070000-0x20012207ffff 64bit pref] [ 0.365186] pci 0000:bd:00.3: reg 0x18: [mem 0x200120700000-0x2001207fffff 64bit pref] [ 0.365212] pci 0000:bd:00.3: reg 0x224: [mem 0x200122000000-0x20012200ffff 64bit pref] [ 0.365216] pci 0000:bd:00.3: VF(n) BAR0 space: [mem 0x200122000000-0x20012206ffff 64bit pref] (contains BAR0 for 7 VFs) [ 0.365221] pci 0000:bd:00.3: reg 0x22c: [mem 0x200120000000-0x2001200fffff 64bit pref] [ 0.365224] pci 0000:bd:00.3: VF(n) BAR2 space: [mem 0x200120000000-0x2001206fffff 64bit pref] (contains BAR2 for 7 VFs) [ 0.365281] pci_bus 0000:bc: on NUMA node 2 [ 0.365285] pci 0000:bc:00.0: bridge window [mem 0x00100000-0x005fffff 64bit pref] to [bus bd] add_size 1d00000 add_align 100000 [ 0.365291] pci 0000:bc:00.0: BAR 15: assigned [mem 0x200120000000-0x2001221fffff 64bit pref] [ 0.365300] pci 0000:bd:00.0: BAR 2: assigned [mem 0x200120000000-0x2001200fffff 64bit pref] [ 0.365305] pci 0000:bd:00.0: BAR 9: assigned [mem 0x200120100000-0x2001207fffff 64bit pref] [ 0.365309] pci 0000:bd:00.1: BAR 2: assigned [mem 0x200120800000-0x2001208fffff 64bit pref] [ 0.365313] pci 0000:bd:00.1: BAR 9: assigned [mem 0x200120900000-0x200120ffffff 64bit pref] [ 0.365317] pci 0000:bd:00.2: BAR 2: assigned [mem 0x200121000000-0x2001210fffff 64bit pref] [ 0.365322] pci 0000:bd:00.2: BAR 9: assigned [mem 0x200121100000-0x2001217fffff 64bit pref] [ 0.365326] pci 0000:bd:00.3: BAR 2: assigned [mem 0x200121800000-0x2001218fffff 64bit pref] [ 0.365330] pci 0000:bd:00.3: BAR 9: assigned [mem 0x200121900000-0x200121ffffff 64bit pref] [ 0.365334] pci 0000:bd:00.0: BAR 0: assigned [mem 0x200122000000-0x20012200ffff 64bit pref] [ 0.365339] pci 0000:bd:00.0: BAR 7: assigned [mem 0x200122010000-0x20012207ffff 64bit pref] [ 0.365342] pci 0000:bd:00.1: BAR 0: assigned [mem 0x200122080000-0x20012208ffff 64bit pref] [ 0.365347] pci 0000:bd:00.1: BAR 7: assigned [mem 0x200122090000-0x2001220fffff 64bit pref] [ 0.365351] pci 0000:bd:00.2: BAR 0: assigned [mem 0x200122100000-0x20012210ffff 64bit pref] [ 0.365356] pci 0000:bd:00.2: BAR 7: assigned [mem 0x200122110000-0x20012217ffff 64bit pref] [ 0.365360] pci 0000:bd:00.3: BAR 0: assigned [mem 0x200122180000-0x20012218ffff 64bit pref] [ 0.365364] pci 0000:bd:00.3: BAR 7: assigned [mem 0x200122190000-0x2001221fffff 64bit pref] [ 0.365370] pci 0000:bc:00.0: PCI bridge to [bus bd] [ 0.365374] pci 0000:bc:00.0: bridge window [mem 0x200120000000-0x2001221fffff 64bit pref] [ 0.365378] pci_bus 0000:bc: resource 4 [mem 0x200120000000-0x20013fffffff pref window] [ 0.365381] pci_bus 0000:bd: resource 2 [mem 0x200120000000-0x2001221fffff 64bit pref] [ 0.365434] ACPI: PCI Root Bridge [PCIB] (domain 0000 [bus b4-b5]) [ 0.365439] acpi PNP0A08:0b: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3] [ 0.365510] acpi PNP0A08:0b: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME AER LTR DPC] [ 0.365573] acpi PNP0A08:0b: _OSC: OS now controls [PCIeCapability] [ 0.366058] acpi PNP0A08:0b: ECAM area [mem 0xdb400000-0xdb5fffff] reserved by PNP0C02:00 [ 0.366065] acpi PNP0A08:0b: ECAM at [mem 0xdb400000-0xdb5fffff] for [bus b4-b5] [ 0.366156] PCI host bridge to bus 0000:b4 [ 0.366160] pci_bus 0000:b4: root bus resource [mem 0x200141000000-0x200141ffffff pref window] [ 0.366163] pci_bus 0000:b4: root bus resource [mem 0x200144000000-0x200145ffffff pref window] [ 0.366166] pci_bus 0000:b4: root bus resource [mem 0xa3000000-0xa3ffffff window] [ 0.366169] pci_bus 0000:b4: root bus resource [bus b4-b5] [ 0.366178] pci 0000:b4:01.0: [19e5:a121] type 01 class 0x060400 [ 0.366188] pci 0000:b4:01.0: enabling Extended Tags [ 0.366260] pci 0000:b4:02.0: [19e5:a230] type 00 class 0x010700 [ 0.366271] pci 0000:b4:02.0: reg 0x24: [mem 0xa3008000-0xa300ffff] [ 0.366356] pci 0000:b4:03.0: [19e5:a235] type 00 class 0x010601 [ 0.366370] pci 0000:b4:03.0: reg 0x24: [mem 0xa3010000-0xa3010fff] [ 0.366423] pci 0000:b4:04.0: [19e5:a230] type 00 class 0x010700 [ 0.366433] pci 0000:b4:04.0: reg 0x24: [mem 0xa3000000-0xa3007fff] [ 0.366543] pci_bus 0000:b6: busn_res: can not insert [bus b6] under [bus b4-b5] (conflicts with (null) [bus b4-b5]) [ 0.366551] pci_bus 0000:b4: on NUMA node 2 [ 0.366555] pci 0000:b4:02.0: BAR 5: assigned [mem 0xa3000000-0xa3007fff] [ 0.366559] pci 0000:b4:04.0: BAR 5: assigned [mem 0xa3008000-0xa300ffff] [ 0.366562] pci 0000:b4:03.0: BAR 5: assigned [mem 0xa3010000-0xa3010fff] [ 0.366566] pci 0000:b4:01.0: PCI bridge to [bus b6] [ 0.366571] pci_bus 0000:b4: resource 4 [mem 0x200141000000-0x200141ffffff pref window] [ 0.366574] pci_bus 0000:b4: resource 5 [mem 0x200144000000-0x200145ffffff pref window] [ 0.366577] pci_bus 0000:b4: resource 6 [mem 0xa3000000-0xa3ffffff window] [ 0.375118] iommu: Default domain type: Translated [ 0.375185] pci 0000:06:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none [ 0.375195] pci 0000:06:00.0: vgaarb: bridge control possible [ 0.375200] pci 0000:06:00.0: vgaarb: setting as boot device (VGA legacy resources not available) [ 0.375202] vgaarb: loaded [ 0.375436] SCSI subsystem initialized [ 0.375466] ACPI: bus type USB registered [ 0.375490] usbcore: registered new interface driver usbfs [ 0.375502] usbcore: registered new interface driver hub [ 0.375556] usbcore: registered new device driver usb [ 0.375575] pps_core: LinuxPPS API ver. 1 registered [ 0.375577] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.375582] PTP clock support registered [ 0.375775] EDAC MC: Ver: 3.0.0 [ 0.375939] Registered efivars operations [ 0.376085] ACPI: arm,spe-v1: must be homogeneous [ 0.376087] ACPI: SPE: Unable to register device [ 0.377271] NetLabel: Initializing [ 0.377274] NetLabel: domain hash size = 128 [ 0.377276] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.377289] NetLabel: unlabeled traffic allowed by default [ 0.377416] sdei: SDEIv1.0 (0x0) detected in firmware. [ 0.379695] clocksource: Switched to clocksource arch_sys_counter [ 0.397537] VFS: Disk quotas dquot_6.6.0 [ 0.397570] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.397809] pnp: PnP ACPI init [ 0.398361] system 00:00: [mem 0xd0000000-0xdfffffff] could not be reserved [ 0.398372] system 00:00: Plug and Play ACPI device, IDs PNP0c02 (active) [ 0.399592] pnp 00:01: Plug and Play ACPI device, IDs PNP0501 (active) [ 0.400786] pnp: PnP ACPI: found 2 devices [ 0.402161] NET: Registered protocol family 2 [ 0.402459] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, vmalloc) [ 0.404835] tcp_listen_portaddr_hash hash table entries: 65536 (order: 8, 1048576 bytes, vmalloc) [ 0.405217] TCP established hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.405739] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, vmalloc) [ 0.405902] TCP: Hash tables configured (established 524288 bind 65536) [ 0.406164] UDP hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.406444] UDP-Lite hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.406833] NET: Registered protocol family 1 [ 0.406868] pci 0000:7a:00.0: enabling device (0000 -> 0002) [ 0.406903] pci 0000:7a:02.0: enabling device (0000 -> 0002) [ 0.406965] pci 0000:ba:00.0: enabling device (0000 -> 0002) [ 0.407005] pci 0000:ba:02.0: enabling device (0000 -> 0002) [ 0.407032] PCI: CLS 32 bytes, default 64 [ 0.407074] Trying to unpack rootfs image as initramfs... [ 0.850625] Freeing initrd memory: 32392K [ 0.854809] hw perfevents: enabled with armv8_pmuv3_0 PMU driver, 13 counters available [ 0.854824] kvm [1]: detected: Hisi CPU type 'Hisi1620' [ 0.854837] kvm [1]: KVM ncsnp enabled [ 0.854839] kvm [1]: KVM dvmbm disabled [ 0.855014] kvm [1]: IPA Size Limit: 48 bits [ 0.855051] kvm [1]: GICv4 support disabled [ 0.855054] kvm [1]: vgic-v2@9b020000 [ 0.855073] kvm [1]: GIC system register CPU interface enabled [ 0.855902] kvm [1]: vgic interrupt IRQ9 [ 0.856840] kvm [1]: VHE mode initialized successfully [ 0.856902] kvm [1]: Shadow device disabled [ 0.858913] Initialise system trusted keyrings [ 0.858930] Key type blacklist registered [ 0.859107] workingset: timestamp_bits=39 max_order=27 bucket_order=0 [ 0.860505] zbud: loaded [ 0.860970] integrity: Platform Keyring initialized [ 0.873140] NET: Registered protocol family 38 [ 0.873145] Key type asymmetric registered [ 0.873148] Asymmetric key parser 'x509' registered [ 0.873151] Asymmetric key parser 'pgp' registered [ 0.873161] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 0.873254] io scheduler mq-deadline registered [ 0.873257] io scheduler kyber registered [ 0.873308] io scheduler bfq registered [ 0.875304] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0x7c00 [ 0.875309] ACPI: IORT: [Firmware Bug]: applying workaround. [ 0.875365] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0xbc00 [ 0.875368] ACPI: IORT: [Firmware Bug]: applying workaround. [ 0.875473] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0x7c00 [ 0.875476] ACPI: IORT: [Firmware Bug]: applying workaround. [ 0.875502] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0xbc00 [ 0.875505] ACPI: IORT: [Firmware Bug]: applying workaround. [ 0.875520] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 0.875777] efifb: probing for efifb [ 0.875802] efifb: framebuffer at 0xe4000000, using 1200k, total 1200k [ 0.875805] efifb: mode is 640x480x32, linelength=2560, pages=1 [ 0.875807] efifb: scrolling: redraw [ 0.875810] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0 [ 0.885043] Console: switching to colour frame buffer device 80x30 [ 0.899646] fb0: EFI VGA frame buffer device [ 0.905845] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0 [ 0.917283] ACPI: Power Button [PWRB] [ 0.930146] thermal LNXTHERM:00: registered as thermal_zone0 [ 0.935898] ACPI: Thermal Zone [TZ00] (46 C) [ 0.946283] thermal LNXTHERM:01: registered as thermal_zone1 [ 0.952054] ACPI: Thermal Zone [TZ01] (49 C) [ 0.957965] [Firmware Bug]: APEI: Invalid physical address in GAR [0x0/64/0/4/0] [ 0.969252] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010000 [ 0.980709] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010010 [ 0.992604] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010020 [ 1.004438] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010030 [ 1.016478] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010040 [ 1.029019] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010050 [ 1.041611] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010060 [ 1.054753] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010070 [ 1.067970] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010080 [ 1.081314] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x0000000044010090 [ 1.095139] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x00000000440100a0 [ 1.109105] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x00000000440100b0 [ 1.123375] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x00000000440100c0 [ 1.137579] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x00000000440100d0 [ 1.151824] ACPI: [Firmware Bug]: requested region covers kernel memory @ 0x00000000440100e0 [ 1.166270] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC. [ 1.180497] ACPI GTDT: found 1 SBSA generic Watchdog(s). [ 1.187813] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 1.215428] 00:01: ttyS0 at MMIO 0x3f00002f8 (irq = 23, base_baud = 115200) is a 16550A [ 10.714413] printk: console [ttyS0] enabled [ 10.727073] arm-smmu-v3 arm-smmu-v3.0.auto: option mask 0x0 [ 10.740170] arm-smmu-v3 arm-smmu-v3.0.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 10.762139] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 65536 entries for cmdq [ 10.783088] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 32768 entries for evtq [ 10.804050] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 65536 entries for priq [ 10.825270] arm-smmu-v3 arm-smmu-v3.1.auto: option mask 0x0 [ 10.838323] arm-smmu-v3 arm-smmu-v3.1.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 10.859922] arm-smmu-v3 arm-smmu-v3.1.auto: allocated 65536 entries for cmdq [ 10.880615] arm-smmu-v3 arm-smmu-v3.1.auto: allocated 32768 entries for evtq [ 10.901116] arm-smmu-v3 arm-smmu-v3.1.auto: allocated 65536 entries for priq [ 10.921812] arm-smmu-v3 arm-smmu-v3.2.auto: option mask 0x0 [ 10.934519] arm-smmu-v3 arm-smmu-v3.2.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 10.955882] arm-smmu-v3 arm-smmu-v3.2.auto: allocated 65536 entries for cmdq [ 10.976496] arm-smmu-v3 arm-smmu-v3.2.auto: allocated 32768 entries for evtq [ 10.997221] arm-smmu-v3 arm-smmu-v3.2.auto: allocated 65536 entries for priq [ 11.018647] arm-smmu-v3 arm-smmu-v3.3.auto: option mask 0x0 [ 11.031636] arm-smmu-v3 arm-smmu-v3.3.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 11.053551] arm-smmu-v3 arm-smmu-v3.3.auto: allocated 65536 entries for cmdq [ 11.074421] arm-smmu-v3 arm-smmu-v3.3.auto: allocated 32768 entries for evtq [ 11.095213] arm-smmu-v3 arm-smmu-v3.3.auto: allocated 65536 entries for priq [ 11.116562] arm-smmu-v3 arm-smmu-v3.4.auto: option mask 0x0 [ 11.129572] arm-smmu-v3 arm-smmu-v3.4.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 11.151543] arm-smmu-v3 arm-smmu-v3.4.auto: allocated 65536 entries for cmdq [ 11.172334] arm-smmu-v3 arm-smmu-v3.4.auto: allocated 32768 entries for evtq [ 11.193147] arm-smmu-v3 arm-smmu-v3.4.auto: allocated 65536 entries for priq [ 11.214537] arm-smmu-v3 arm-smmu-v3.5.auto: option mask 0x0 [ 11.227517] arm-smmu-v3 arm-smmu-v3.5.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 11.249428] arm-smmu-v3 arm-smmu-v3.5.auto: allocated 65536 entries for cmdq [ 11.270210] arm-smmu-v3 arm-smmu-v3.5.auto: allocated 32768 entries for evtq [ 11.291030] arm-smmu-v3 arm-smmu-v3.5.auto: allocated 65536 entries for priq [ 11.312340] arm-smmu-v3 arm-smmu-v3.6.auto: option mask 0x0 [ 11.325319] arm-smmu-v3 arm-smmu-v3.6.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 11.347265] arm-smmu-v3 arm-smmu-v3.6.auto: allocated 65536 entries for cmdq [ 11.368043] arm-smmu-v3 arm-smmu-v3.6.auto: allocated 32768 entries for evtq [ 11.388895] arm-smmu-v3 arm-smmu-v3.6.auto: allocated 65536 entries for priq [ 11.410252] arm-smmu-v3 arm-smmu-v3.7.auto: option mask 0x0 [ 11.423249] arm-smmu-v3 arm-smmu-v3.7.auto: ias 48-bit, oas 48-bit (features 0x00071fff) [ 11.445204] arm-smmu-v3 arm-smmu-v3.7.auto: allocated 65536 entries for cmdq [ 11.466002] arm-smmu-v3 arm-smmu-v3.7.auto: allocated 32768 entries for evtq [ 11.486961] arm-smmu-v3 arm-smmu-v3.7.auto: allocated 65536 entries for priq [ 11.517030] virdev: Register virtdev platform driver succeed. [ 11.530197] rdac: device handler registered [ 11.541814] hp_sw: device handler registered [ 11.553238] emc: device handler registered [ 11.564442] alua: device handler registered [ 11.579057] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 11.598499] ehci-pci: EHCI PCI platform driver [ 11.609972] ehci-pci 0000:7a:01.0: Adding to iommu group 0 [ 11.623051] ehci-pci 0000:7a:01.0: EHCI Host Controller [ 11.635024] ehci-pci 0000:7a:01.0: new USB bus registered, assigned bus number 1 [ 11.654694] ehci-pci 0000:7a:01.0: applying Synopsys HC workaround [ 11.667463] ehci-pci 0000:7a:01.0: irq 52, io mem 0x20c101000 [ 11.695659] ehci-pci 0000:7a:01.0: USB 2.0 started, EHCI 1.00 [ 11.707686] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.10 [ 11.727788] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 11.747104] usb usb1: Product: EHCI Host Controller [ 11.758265] usb usb1: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 ehci_hcd [ 11.777975] usb usb1: SerialNumber: 0000:7a:01.0 [ 11.788855] hub 1-0:1.0: USB hub found [ 11.798611] hub 1-0:1.0: 2 ports detected [ 11.808595] ehci-pci 0000:ba:01.0: Adding to iommu group 1 [ 11.820850] ehci-pci 0000:ba:01.0: EHCI Host Controller [ 11.832065] ehci-pci 0000:ba:01.0: new USB bus registered, assigned bus number 2 [ 11.850670] ehci-pci 0000:ba:01.0: applying Synopsys HC workaround [ 11.862997] ehci-pci 0000:ba:01.0: irq 53, io mem 0x20020c101000 [ 11.887664] ehci-pci 0000:ba:01.0: USB 2.0 started, EHCI 1.00 [ 11.899360] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.10 [ 11.918988] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 11.937878] usb usb2: Product: EHCI Host Controller [ 11.949083] usb usb2: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 ehci_hcd [ 11.968958] usb usb2: SerialNumber: 0000:ba:01.0 [ 11.980232] hub 2-0:1.0: USB hub found [ 11.990204] hub 2-0:1.0: 2 ports detected [ 12.000373] ehci-platform: EHCI generic platform driver [ 12.011693] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 12.023894] ohci-pci: OHCI PCI platform driver [ 12.034341] ohci-pci 0000:7a:00.0: Adding to iommu group 2 [ 12.046383] ohci-pci 0000:7a:00.0: OHCI PCI host controller [ 12.057762] ohci-pci 0000:7a:00.0: new USB bus registered, assigned bus number 3 [ 12.076070] ohci-pci 0000:7a:00.0: irq 54, io mem 0x20c100000 [ 12.087831] usb 1-1: new high-speed USB device number 2 using ehci-pci [ 12.151698] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 5.10 [ 12.171644] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.190949] usb usb3: Product: OHCI PCI host controller [ 12.202674] usb usb3: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 ohci_hcd [ 12.222925] usb usb3: SerialNumber: 0000:7a:00.0 [ 12.234178] hub 3-0:1.0: USB hub found [ 12.244195] hub 3-0:1.0: 2 ports detected [ 12.254498] ohci-pci 0000:ba:00.0: Adding to iommu group 3 [ 12.266887] ohci-pci 0000:ba:00.0: OHCI PCI host controller [ 12.278546] ohci-pci 0000:ba:00.0: new USB bus registered, assigned bus number 4 [ 12.297384] ohci-pci 0000:ba:00.0: irq 55, io mem 0x20020c100000 [ 12.328391] usb 1-1: New USB device found, idVendor=0bda, idProduct=5411, bcdDevice= 1.01 [ 12.348499] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 12.367976] usb 1-1: Product: 4-Port USB 2.1 Hub [ 12.371710] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 5.10 [ 12.379295] usb 1-1: Manufacturer: Generic [ 12.400171] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.411173] hub 1-1:1.0: USB hub found [ 12.430545] usb usb4: Product: OHCI PCI host controller [ 12.441398] hub 1-1:1.0: 4 ports detected [ 12.452413] usb usb4: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 ohci_hcd [ 12.482700] usb usb4: SerialNumber: 0000:ba:00.0 [ 12.493770] hub 4-0:1.0: USB hub found [ 12.503768] hub 4-0:1.0: 2 ports detected [ 12.514098] uhci_hcd: USB Universal Host Controller Interface driver [ 12.526977] xhci_hcd 0000:7a:02.0: Adding to iommu group 4 [ 12.539500] xhci_hcd 0000:7a:02.0: xHCI Host Controller [ 12.551309] xhci_hcd 0000:7a:02.0: new USB bus registered, assigned bus number 5 [ 12.570715] xhci_hcd 0000:7a:02.0: hcc params 0x0220f66d hci version 0x100 quirks 0x0000000000000010 [ 12.592217] xhci_hcd 0000:7a:02.0: xHCI Host Controller [ 12.603875] xhci_hcd 0000:7a:02.0: new USB bus registered, assigned bus number 6 [ 12.623063] xhci_hcd 0000:7a:02.0: Host supports USB 3.0 SuperSpeed [ 12.635791] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.10 [ 12.656115] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.675458] usb usb5: Product: xHCI Host Controller [ 12.686829] usb usb5: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 xhci-hcd [ 12.707084] usb usb5: SerialNumber: 0000:7a:02.0 [ 12.718546] hub 5-0:1.0: USB hub found [ 12.728768] hub 5-0:1.0: 1 port detected [ 12.739024] usb usb6: We don't know the algorithms for LPM for this host, disabling LPM. [ 12.759247] usb usb6: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.10 [ 12.780033] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 12.800072] usb usb6: Product: xHCI Host Controller [ 12.811920] usb usb6: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 xhci-hcd [ 12.832973] usb usb6: SerialNumber: 0000:7a:02.0 [ 12.844537] hub 6-0:1.0: USB hub found [ 12.855149] hub 6-0:1.0: 1 port detected [ 12.865847] xhci_hcd 0000:ba:02.0: Adding to iommu group 5 [ 12.878640] xhci_hcd 0000:ba:02.0: xHCI Host Controller [ 12.890490] xhci_hcd 0000:ba:02.0: new USB bus registered, assigned bus number 7 [ 12.909884] xhci_hcd 0000:ba:02.0: hcc params 0x0220f66d hci version 0x100 quirks 0x0000000000000010 [ 12.931140] xhci_hcd 0000:ba:02.0: xHCI Host Controller [ 12.942808] xhci_hcd 0000:ba:02.0: new USB bus registered, assigned bus number 8 [ 12.943660] usb 1-1.1: new full-speed USB device number 3 using ehci-pci [ 12.962058] xhci_hcd 0000:ba:02.0: Host supports USB 3.0 SuperSpeed [ 12.993774] usb usb7: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.10 [ 13.014620] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 13.034555] usb usb7: Product: xHCI Host Controller [ 13.046034] usb usb7: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 xhci-hcd [ 13.066431] usb usb7: SerialNumber: 0000:ba:02.0 [ 13.077777] hub 7-0:1.0: USB hub found [ 13.088021] hub 7-0:1.0: 1 port detected [ 13.098326] usb usb8: We don't know the algorithms for LPM for this host, disabling LPM. [ 13.111655] usb 5-1: new high-speed USB device number 2 using xhci_hcd [ 13.118688] usb usb8: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.10 [ 13.159419] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 13.179768] usb usb8: Product: xHCI Host Controller [ 13.184769] usb 1-1.1: New USB device found, idVendor=12d1, idProduct=0003, bcdDevice= 1.00 [ 13.191628] usb usb8: Manufacturer: Linux 5.10.0-188.0.0.101.oe2203sp3.aarch64 xhci-hcd [ 13.191631] usb usb8: SerialNumber: 0000:ba:02.0 [ 13.213291] usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 13.213294] usb 1-1.1: Product: Keyboard/Mouse KVM 1.1.0 [ 13.234952] hub 8-0:1.0: USB hub found [ 13.290189] hub 8-0:1.0: 1 port detected [ 13.300970] mousedev: PS/2 mouse device common for all mice [ 13.329908] rtc-efi rtc-efi.0: registered as rtc0 [ 13.338251] usb 5-1: New USB device found, idVendor=0bda, idProduct=5411, bcdDevice= 1.01 [ 13.362293] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 13.362296] rtc-efi rtc-efi.0: setting system clock to 2024-03-05T09:46:01 UTC (1709631961) [ 13.382712] usb 5-1: Product: 4-Port USB 2.1 Hub [ 13.405309] hid: raw HID events driver (C) Jiri Kosina [ 13.416119] usb 5-1: Manufacturer: Generic [ 13.446976] input: Keyboard/Mouse KVM 1.1.0 as /devices/pci0000:7a/0000:7a:01.0/usb1/1-1/1-1.1/1-1.1:1.0/0003:12D1:0003.0001/input/input1 [ 13.467878] hub 5-1:1.0: USB hub found [ 13.483126] usb 6-1: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd [ 13.503278] hub 5-1:1.0: 4 ports detected [ 13.528177] usb 6-1: New USB device found, idVendor=0bda, idProduct=0411, bcdDevice= 1.01 [ 13.531797] hid-generic 0003:12D1:0003.0001: input,hidraw0: USB HID v1.10 Keyboard [Keyboard/Mouse KVM 1.1.0] on usb-0000:7a:01.0-1.1/input0 [ 13.548971] usb 6-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [ 13.582036] input: Keyboard/Mouse KVM 1.1.0 as /devices/pci0000:7a/0000:7a:01.0/usb1/1-1/1-1.1/1-1.1:1.1/0003:12D1:0003.0002/input/input2 [ 13.594743] usb 6-1: Product: 4-Port USB 3.1 Hub [ 13.620945] hid-generic 0003:12D1:0003.0002: input,hidraw1: USB HID v1.10 Mouse [Keyboard/Mouse KVM 1.1.0] on usb-0000:7a:01.0-1.1/input1 [ 13.632771] usb 6-1: Manufacturer: Generic [ 13.659127] usbcore: registered new interface driver usbhid [ 13.683224] usbhid: USB HID core driver [ 13.694248] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.8.auto: option mask 0x3 [ 13.707843] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.8.auto: Registered PMU @ 0x0000000148020000 using 8 counters with Individual filter settings [ 13.710230] hub 6-1:1.0: USB hub found [ 13.733652] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.9.auto: option mask 0x3 [ 13.745115] hub 6-1:1.0: 4 ports detected [ 13.758011] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.9.auto: Registered PMU @ 0x0000000100020000 using 8 counters with Individual filter settings [ 13.794935] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.10.auto: option mask 0x3 [ 13.815026] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.10.auto: Registered PMU @ 0x0000000140020000 using 8 counters with Individual filter settings [ 13.841590] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.11.auto: option mask 0x3 [ 13.861925] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.11.auto: Registered PMU @ 0x0000200148020000 using 8 counters with Individual filter settings [ 13.888418] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.12.auto: option mask 0x3 [ 13.908876] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.12.auto: Registered PMU @ 0x0000200100020000 using 8 counters with Individual filter settings [ 13.935457] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.13.auto: option mask 0x3 [ 13.955912] arm-smmu-v3-pmcg arm-smmu-v3-pmcg.13.auto: Registered PMU @ 0x0000200140020000 using 8 counters with Individual filter settings [ 13.983307] Initializing XFRM netlink socket [ 13.995330] NET: Registered protocol family 10 [ 14.007930] Segment Routing with IPv6 [ 14.019086] NET: Registered protocol family 17 [ 14.031239] registered taskstats version 1 [ 14.043750] Loading compiled-in X.509 certificates [ 14.099184] Loaded X.509 cert 'openEuler kernel signing key: 381049b4ab0317fa4a59c3945f7dd11f8aca835b' [ 14.122207] Load PGP public keys [ 14.132435] Loaded PGP key 'openeuler fb37bc6f' [ 14.152527] Loaded PGP key 'private OBS b25e7f66' [ 14.170280] cryptd: max_cpu_qlen set to 1000 [ 14.224950] Key type encrypted registered [ 14.235686] integrity: Loading X.509 certificate: UEFI:db [ 14.247726] integrity: Loaded X.509 cert 'Huawei Technologies Co., Ltd.: EulerOS Code Signing Certificate: 0f081cc1d2fedc52749211b644da32895618698f' [ 14.273518] integrity: Loading X.509 certificate: UEFI:MokListRT (MOKvar table) [ 14.293906] integrity: Loaded X.509 cert 'openEuler: CA: 1e1baa926835cdd94919d433c34401a2e5f53df9' [ 14.315072] ima: No TPM chip found, activating TPM-bypass! [ 14.327026] ima: Allocated hash algorithm: sha256 [ 14.338074] ima: No architecture policies found [ 14.348793] evm: Initialising EVM extended attributes: [ 14.360037] evm: security.selinux [ 14.369389] evm: security.apparmor [ 14.378737] evm: security.ima [ 14.387542] evm: security.capability [ 14.396894] evm: HMAC attrs: 0x1 [ 14.406467] pcieport 0000:00:00.0: Adding to iommu group 6 [ 14.419135] pcieport 0000:00:00.0: PME: Signaling with IRQ 64 [ 14.430626] pcieport 0000:00:00.0: AER: enabled with IRQ 65 [ 14.441852] pcieport 0000:00:04.0: Adding to iommu group 7 [ 14.453893] pcieport 0000:00:04.0: PME: Signaling with IRQ 66 [ 14.465048] pcieport 0000:00:04.0: AER: enabled with IRQ 67 [ 14.475940] pcieport 0000:00:08.0: Adding to iommu group 8 [ 14.487917] pcieport 0000:00:08.0: PME: Signaling with IRQ 68 [ 14.498919] pcieport 0000:00:08.0: AER: enabled with IRQ 69 [ 14.509650] pcieport 0000:00:0c.0: Adding to iommu group 9 [ 14.521357] pcieport 0000:00:0c.0: PME: Signaling with IRQ 70 [ 14.532142] pcieport 0000:00:0c.0: AER: enabled with IRQ 71 [ 14.542485] pcieport 0000:00:10.0: Adding to iommu group 10 [ 14.553860] pcieport 0000:00:10.0: PME: Signaling with IRQ 72 [ 14.564097] pcieport 0000:00:10.0: AER: enabled with IRQ 73 [ 14.573984] pcieport 0000:00:11.0: Adding to iommu group 11 [ 14.584735] pcieport 0000:00:11.0: PME: Signaling with IRQ 74 [ 14.594639] pcieport 0000:00:11.0: AER: enabled with IRQ 75 [ 14.604311] pcieport 0000:00:12.0: Adding to iommu group 12 [ 14.615005] pcieport 0000:00:12.0: PME: Signaling with IRQ 76 [ 14.624793] pcieport 0000:00:12.0: AER: enabled with IRQ 77 [ 14.634603] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0x7c00 [ 14.651787] ACPI: IORT: [Firmware Bug]: applying workaround. [ 14.662076] pcieport 0000:7c:00.0: Adding to iommu group 13 [ 14.672830] pcieport 0000:74:01.0: Adding to iommu group 14 [ 14.683440] pcieport 0000:80:00.0: Adding to iommu group 15 [ 14.694715] pcieport 0000:80:00.0: PME: Signaling with IRQ 78 [ 14.705183] pcieport 0000:80:00.0: AER: enabled with IRQ 79 [ 14.715282] pcieport 0000:80:08.0: Adding to iommu group 16 [ 14.726506] pcieport 0000:80:08.0: PME: Signaling with IRQ 80 [ 14.736927] pcieport 0000:80:08.0: AER: enabled with IRQ 81 [ 14.746961] pcieport 0000:80:08.0: pciehp: Slot #28 AttnBtn- PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock- NoCompl- IbPresDis- LLActRep- [ 14.773087] pcieport 0000:80:0a.0: Adding to iommu group 17 [ 14.784789] pcieport 0000:80:0a.0: PME: Signaling with IRQ 82 [ 14.795566] pcieport 0000:80:0a.0: AER: enabled with IRQ 83 [ 14.806219] pcieport 0000:80:0a.0: pciehp: Slot #30 AttnBtn- PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock- NoCompl- IbPresDis- LLActRep- [ 14.833728] pcieport 0000:80:0c.0: Adding to iommu group 18 [ 14.845802] pcieport 0000:80:0c.0: PME: Signaling with IRQ 84 [ 14.857056] pcieport 0000:80:0c.0: AER: enabled with IRQ 85 [ 14.868030] pcieport 0000:80:0c.0: pciehp: Slot #32 AttnBtn- PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock- NoCompl- IbPresDis- LLActRep- [ 14.896546] pcieport 0000:80:0e.0: Adding to iommu group 19 [ 14.908873] pcieport 0000:80:0e.0: PME: Signaling with IRQ 86 [ 14.920096] pcieport 0000:80:0e.0: AER: enabled with IRQ 87 [ 14.931143] pcieport 0000:80:0e.0: pciehp: Slot #34 AttnBtn- PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock- NoCompl- IbPresDis- LLActRep- [ 14.960522] pcieport 0000:80:10.0: Adding to iommu group 20 [ 14.973276] pcieport 0000:80:10.0: PME: Signaling with IRQ 88 [ 14.985141] pcieport 0000:80:10.0: AER: enabled with IRQ 89 [ 14.996777] ACPI: IORT: [Firmware Bug]: [map (____ptrval____)] conflicting mapping for input ID 0xbc00 [ 15.017719] ACPI: IORT: [Firmware Bug]: applying workaround. [ 15.029499] pcieport 0000:bc:00.0: Adding to iommu group 21 [ 15.041840] pcieport 0000:b4:01.0: Adding to iommu group 22 [ 15.057810] SDEI NMI watchdog: SDEI Watchdog registered successfully [ 15.071716] integrity: Unable to open file: /etc/keys/x509_ima.der (-2) [ 15.071719] integrity: Unable to open file: /etc/keys/x509_evm.der (-2) [ 15.091893] Freeing unused kernel memory: 4608K [ 15.143705] Run /init as init process [ 15.153527] with arguments: [ 15.153528] /init [ 15.153529] with environment: [ 15.153530] HOME=/ [ 15.153531] TERM=linux [ 15.153532] BOOT_IMAGE=/vmlinuz-5.10.0-188.0.0.101.oe2203sp3.aarch64 [ 15.153532] crashkernel=1024M,high [ 15.176358] systemd[1]: systemd v249-63.oe2203sp3 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA -SMACK +SECCOMP +GCRYPT +GNUTLS -OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB -ZSTD +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=legacy) [ 15.238059] systemd[1]: Detected architecture arm64. [ 15.249662] systemd[1]: Running in initial RAM disk. [ 15.333231] systemd[1]: Hostname set to . [ 15.409686] systemd[1]: Queued start job for default target Initrd Default Target. [ 15.429999] random: systemd: uninitialized urandom read (16 bytes read) [ 15.449236] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ 15.476673] random: systemd: uninitialized urandom read (16 bytes read) [ 15.496616] systemd[1]: Reached target Local File Systems. [ 15.515693] random: systemd: uninitialized urandom read (16 bytes read) [ 15.535848] systemd[1]: Reached target Path Units. [ 15.554265] systemd[1]: Reached target Slice Units. [ 15.572643] systemd[1]: Reached target Swaps. [ 15.590006] systemd[1]: Reached target Timer Units. [ 15.608261] systemd[1]: Listening on Journal Socket (/dev/log). [ 15.627658] systemd[1]: Listening on Journal Socket. [ 15.646146] systemd[1]: Listening on udev Control Socket. [ 15.665061] systemd[1]: Listening on udev Kernel Socket. [ 15.683740] systemd[1]: Reached target Socket Units. [ 15.749462] systemd[1]: Starting Create List of Static Device Nodes... [ 15.776319] systemd[1]: Started Hardware RNG Entropy Gatherer Daemon. [ 15.796491] systemd[1]: systemd-journald.service: unit configures an IP firewall, but the local system does not support BPF/cgroup firewalling. [ 15.822820] systemd[1]: (This warning is only shown for the first unit using IP firewalling.) [ 15.846314] systemd[1]: Starting Journal Service... [ 15.865861] systemd[1]: Starting Load Kernel Modules... [ 15.870174] xpmem: loading out-of-tree module taints kernel. [ 15.891374] xpmem: module verification failed: signature and/or required key missing - tainting kernel [ 15.916136] XPMEM kernel module v2.7.3 loaded [ 15.933550] systemd[1]: Starting Setup Virtual Console... [ 15.953916] systemd[1]: Finished Create List of Static Device Nodes. [ 15.983698] systemd[1]: Started Journal Service. [ 16.187791] device-mapper: uevent: version 1.0.3 [ 16.199689] device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialised: dm-devel@redhat.com [ 16.436092] Compat-mlnx-ofed backport release: a675be0 [ 16.448127] Backport based on mlnx_ofed/mlnx-ofa_kernel-4.0.git a675be0 [ 16.455024] random: systemd: uninitialized urandom read (16 bytes read) [ 16.467543] compat.git: mlnx_ofed/mlnx-ofa_kernel-4.0.git [ 16.475106] libata version 3.00 loaded. [ 16.475182] edma_drv 0000:05:00.0: Adding to iommu group 23 [ 16.475787] edma_drv 0000:05:00.0: enabling device (0140 -> 0142) [ 16.476925] megasas: 07.714.04.00-rc1 [ 16.477010] megaraid_sas 0000:03:00.0: Adding to iommu group 24 [ 16.478006] megaraid_sas 0000:03:00.0: BAR:0x0 BAR's base_addr(phys):0x0000080000e00000 mapped virt_addr:0x(____ptrval____) [ 16.478010] megaraid_sas 0000:03:00.0: FW now in Ready state [ 16.478012] megaraid_sas 0000:03:00.0: 63 bit DMA mask and 63 bit consistent mask [ 16.478288] megaraid_sas 0000:03:00.0: firmware supports msix : (128) [ 16.480307] megaraid_sas 0000:03:00.0: requested/available msix 97/97 [ 16.480322] megaraid_sas 0000:03:00.0: current msix/online cpus : (97/96) [ 16.480324] megaraid_sas 0000:03:00.0: RDPQ mode : (enabled) [ 16.480326] megaraid_sas 0000:03:00.0: Current firmware supports maximum commands: 9197 LDIO threshold: 0 [ 16.487244] random: systemd: uninitialized urandom read (16 bytes read) [ 16.494269] sbsa-gwdt sbsa-gwdt.0: Initialized with 10s timeout @ 100000000 Hz, action=0. [ 16.494703] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver [ 16.494705] ixgbe: Copyright (c) 1999-2016 Intel Corporation. [ 16.495012] ixgbe 0000:01:00.0: Adding to iommu group 25 [ 16.506299] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version [ 16.526377] nvme 0000:82:00.0: Adding to iommu group 26 [ 16.535243] hns3: Copyright (c) 2017 Huawei Corporation. [ 16.548640] nvme nvme0: pci function 0000:82:00.0 [ 16.549772] ahci 0000:74:03.0: Adding to iommu group 27 [ 16.560144] hns3 0000:7d:00.0: Adding to iommu group 28 [ 16.580001] nvme 0000:83:00.0: Adding to iommu group 29 [ 16.590048] nvme nvme0: Shutdown timeout set to 8 seconds [ 16.598176] nvme nvme0: 64/0/0 default/read/poll queues [ 16.607089] nvme nvme1: pci function 0000:83:00.0 [ 16.876649] nvme 0000:84:00.0: Adding to iommu group 30 [ 16.877271] nvme nvme2: pci function 0000:84:00.0 [ 16.887128] nvme nvme1: Shutdown timeout set to 8 seconds [ 16.895136] nvme nvme1: 64/0/0 default/read/poll queues [ 16.932484] nvme 0000:85:00.0: Adding to iommu group 31 [ 16.933126] nvme nvme3: pci function 0000:85:00.0 [ 16.943649] nvme nvme3: Shutdown timeout set to 8 seconds [ 16.943931] nvme nvme2: Shutdown timeout set to 8 seconds [ 16.952016] nvme nvme2: 64/0/0 default/read/poll queues [ 16.971159] nvme nvme3: 64/0/0 default/read/poll queues [ 17.012937] random: systemd: uninitialized urandom read (16 bytes read) [ 17.084774] megaraid_sas 0000:03:00.0: Performance mode :Latency (latency index = 1) [ 17.104575] megaraid_sas 0000:03:00.0: FW supports sync cache : Yes [ 17.104581] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x40000009 [ 17.139293] ahci 0000:74:03.0: version 3.0 [ 17.139313] ahci 0000:74:03.0: controller does not support SXS, disabling CAP_SXS [ 17.179710] ahci 0000:74:03.0: SSS flag set, parallel bus scan disabled [ 17.199301] ahci 0000:74:03.0: AHCI 0001.0300 32 slots 2 ports 6 Gbps 0x3 impl SATA mode [ 17.220904] ahci 0000:74:03.0: flags: 64bit ncq sntf stag pm led clo only pmp fbs slum part ccc ems boh [ 17.244634] hns3 0000:7d:00.1: Adding to iommu group 32 [ 17.257701] hns3 0000:7d:00.2: Adding to iommu group 33 [ 17.270566] hns3 0000:7d:00.3: Adding to iommu group 34 [ 17.270689] hisi_sas_v3_hw 0000:74:02.0: Adding to iommu group 35 [ 17.283245] hns3 0000:bd:00.0: Adding to iommu group 36 [ 17.296275] hisi_sas_v3_hw 0000:74:02.0: enabling device (0000 -> 0002) [ 17.308546] hns3 0000:bd:00.1: Adding to iommu group 37 [ 17.314425] mlx5_core 0000:81:00.0: Adding to iommu group 38 [ 17.315476] mlx5_core 0000:81:00.0: firmware version: 16.31.1014 [ 17.315515] mlx5_core 0000:81:00.0: 126.016 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x16 link) [ 17.347689] scsi host1: ahci [ 17.353890] hns3 0000:bd:00.2: Adding to iommu group 39 [ 17.389141] hisi_sas_v3_hw 0000:74:02.0: 16 hw queues [ 17.390360] hns3 0000:bd:00.3: Adding to iommu group 40 [ 17.399953] scsi host2: hisi_sas_v3_hw [ 17.451496] hclge is initializing [ 17.455694] scsi host3: ahci [ 17.462127] hns3 0000:7d:00.0: The firmware version is 1.10.0.6 [ 17.483700] ata1: SATA max UDMA/133 abar m4096@0xa2010000 port 0xa2010100 irq 448 [ 17.505946] ata2: SATA max UDMA/133 abar m4096@0xa2010000 port 0xa2010180 irq 449 [ 17.544082] mlx5_core 0000:81:00.0: Port module event: module 0, Cable plugged [ 17.559575] hns3 0000:7d:00.0: hclge driver initialization finished. [ 17.565153] mlx5_core 0000:81:00.0: mlx5_pcie_event:304:(pid 701): PCIe slot advertised sufficient power (27W). [ 17.567797] ahci 0000:b4:03.0: Adding to iommu group 41 [ 17.568403] ahci 0000:b4:03.0: controller does not support SXS, disabling CAP_SXS [ 17.568475] ahci 0000:b4:03.0: SSS flag set, parallel bus scan disabled [ 17.568492] ahci 0000:b4:03.0: AHCI 0001.0300 32 slots 2 ports 6 Gbps 0x3 impl SATA mode [ 17.568496] ahci 0000:b4:03.0: flags: 64bit ncq sntf stag pm led clo only pmp fbs slum part ccc ems boh [ 17.568932] scsi host4: ahci [ 17.569242] scsi host5: ahci [ 17.569334] ata3: SATA max UDMA/133 abar m4096@0xa3010000 port 0xa3010100 irq 676 [ 17.569336] ata4: SATA max UDMA/133 abar m4096@0xa3010000 port 0xa3010180 irq 677 [ 17.590285] hns3 0000:7d:00.1: The firmware version is 1.10.0.6 [ 17.594000] mlx5_core 0000:81:00.1: Adding to iommu group 42 [ 17.595105] mlx5_core 0000:81:00.1: firmware version: 16.31.1014 [ 17.595160] mlx5_core 0000:81:00.1: 126.016 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x16 link) [ 17.838582] mlx5_core 0000:81:00.1: Port module event: module 1, Cable plugged [ 17.841789] ata1: SATA link down (SStatus 0 SControl 300) [ 17.861509] mlx5_core 0000:81:00.1: mlx5_pcie_event:304:(pid 1266): PCIe slot advertised sufficient power (27W). [ 17.881789] ata3: SATA link down (SStatus 0 SControl 300) [ 17.903115] hns3 0000:7d:00.1: hclge driver initialization finished. [ 17.940176] hns3 0000:7d:00.2: The firmware version is 1.10.0.6 [ 18.039591] hns3 0000:7d:00.2: hclge driver initialization finished. [ 18.065106] hns3 0000:7d:00.3: The firmware version is 1.10.0.6 [ 18.183136] hns3 0000:7d:00.3: hclge driver initialization finished. [ 18.189782] ata2: SATA link down (SStatus 0 SControl 300) [ 18.208585] hns3 0000:bd:00.0: The firmware version is 1.10.0.6 [ 18.332028] hns3 0000:bd:00.0: hclge driver initialization finished. [ 18.357753] hns3 0000:bd:00.1: The firmware version is 1.10.0.6 [ 18.459521] hns3 0000:bd:00.1: hclge driver initialization finished. [ 18.485084] hns3 0000:bd:00.2: The firmware version is 1.10.0.6 [ 18.511661] megaraid_sas 0000:03:00.0: FW provided supportMaxExtLDs: 1 max_lds: 240 [ 18.521796] ata4: SATA link down (SStatus 0 SControl 300) [ 18.533084] megaraid_sas 0000:03:00.0: controller type : MR(2048MB) [ 18.559971] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR) : Enabled [ 18.581222] megaraid_sas 0000:03:00.0: Secure JBOD support : Yes [ 18.594718] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes [ 18.604136] hns3 0000:bd:00.2: hclge driver initialization finished. [ 18.608257] megaraid_sas 0000:03:00.0: FW provided TM TaskAbort/Reset timeout : 6 secs/60 secs [ 18.633861] hns3 0000:bd:00.3: The firmware version is 1.10.0.6 [ 18.645004] megaraid_sas 0000:03:00.0: JBOD sequence map support : Yes [ 18.679424] megaraid_sas 0000:03:00.0: PCI Lane Margining support : No [ 18.743540] hns3 0000:bd:00.3: hclge driver initialization finished. [ 18.771281] hns3 0000:7d:00.0 enp125s0f0: renamed from eth0 [ 18.815782] hns3 0000:7d:00.2 enp125s0f2: renamed from eth2 [ 18.864591] hns3 0000:bd:00.0 enp189s0f0: renamed from eth4 [ 18.894896] ixgbe 0000:01:00.0: Multiqueue Enabled: Rx Queue count = 63, Tx Queue count = 63 XDP Queue count = 0 [ 18.919360] ixgbe 0000:01:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 18.942228] ixgbe 0000:01:00.0: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF [ 18.956077] ixgbe 0000:01:00.0: e0:00:84:56:58:3e [ 18.979825] megaraid_sas 0000:03:00.0: NVME page size : (4096) [ 18.994111] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is called outbound_intr_mask:0x40000000 [ 19.003833] hns3 0000:7d:00.3 enp125s0f3: renamed from eth3 [ 19.018456] megaraid_sas 0000:03:00.0: INIT adapter done [ 19.095835] hns3 0000:bd:00.1 enp189s0f1: renamed from eth5 [ 19.115814] megaraid_sas 0000:03:00.0: pci id : (0x1000)/(0x0016)/(0x19e5)/(0xda15) [ 19.138213] megaraid_sas 0000:03:00.0: unevenspan support : no [ 19.151842] megaraid_sas 0000:03:00.0: firmware crash dump : no [ 19.159838] hns3 0000:bd:00.2 enp189s0f2: renamed from eth6 [ 19.165433] megaraid_sas 0000:03:00.0: JBOD sequence map : enabled [ 19.221873] random: crng init done [ 19.232942] random: 68 urandom warning(s) missed due to ratelimiting [ 19.263664] megaraid_sas 0000:03:00.0: Max firmware commands: 9196 shared with nr_hw_queues = 96 [ 19.263887] hns3 0000:7d:00.1 enp125s0f1: renamed from eth1 [ 19.286934] scsi host0: Avago SAS based MegaRAID driver [ 19.347862] hns3 0000:bd:00.3 enp189s0f3: renamed from eth7 [ 19.435191] ixgbe 0000:01:00.0: Intel(R) 10 Gigabit Network Connection [ 19.467803] ixgbe 0000:01:00.1: Adding to iommu group 43 [ 19.553210] scsi 0:3:111:0: Direct-Access AVAGO HW-SAS3508 5.06 PQ: 0 ANSI: 5 [ 19.589601] sd 0:3:111:0: [sda] 3904294912 512-byte logical blocks: (2.00 TB/1.82 TiB) [ 19.612709] sd 0:3:111:0: [sda] Write Protect is off [ 19.625599] sd 0:3:111:0: [sda] Mode Sense: 1f 00 10 08 [ 19.625655] sd 0:3:111:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA [ 19.651615] sd 0:3:111:0: [sda] Optimal transfer size 262144 bytes [ 19.711732] hisi_sas_v3_hw 0000:74:02.0: neither _PS0 nor _PR0 is defined [ 19.738176] sda: sda1 sda2 sda3 [ 19.749496] sd 0:3:111:0: [sda] Attached SCSI disk [ 19.752633] hisi_sas_v3_hw 0000:74:04.0: Adding to iommu group 44 [ 19.776757] hisi_sas_v3_hw 0000:74:04.0: enabling device (0000 -> 0002) [ 19.836573] hisi_sas_v3_hw 0000:74:04.0: 16 hw queues [ 19.849356] scsi host6: hisi_sas_v3_hw [ 21.037711] ixgbe 0000:01:00.1: Multiqueue Enabled: Rx Queue count = 63, Tx Queue count = 63 XDP Queue count = 0 [ 21.063109] ixgbe 0000:01:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link) [ 21.086757] ixgbe 0000:01:00.1: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF [ 21.100977] ixgbe 0000:01:00.1: e0:00:84:56:58:3f [ 21.219665] ixgbe 0000:01:00.1: Intel(R) 10 Gigabit Network Connection [ 21.257040] ixgbe 0000:01:00.1 enp1s0f1: renamed from eth1 [ 21.292133] ixgbe 0000:01:00.0 enp1s0f0: renamed from eth0 [ 21.691719] hisi_sas_v3_hw 0000:74:04.0: neither _PS0 nor _PR0 is defined [ 21.728797] hisi_sas_v3_hw 0000:b4:02.0: Adding to iommu group 45 [ 21.743104] hisi_sas_v3_hw 0000:b4:02.0: enabling device (0000 -> 0002) [ 21.774024] hisi_sas_v3_hw 0000:b4:02.0: 16 hw queues [ 21.786620] scsi host7: hisi_sas_v3_hw [ 22.987722] hisi_sas_v3_hw 0000:b4:02.0: neither _PS0 nor _PR0 is defined [ 23.009881] hisi_sas_v3_hw 0000:b4:04.0: Adding to iommu group 46 [ 23.023951] hisi_sas_v3_hw 0000:b4:04.0: enabling device (0000 -> 0002) [ 23.054488] hisi_sas_v3_hw 0000:b4:04.0: 16 hw queues [ 23.067065] scsi host8: hisi_sas_v3_hw [ 24.271739] hisi_sas_v3_hw 0000:b4:04.0: neither _PS0 nor _PR0 is defined [ 25.378869] device-mapper: ioctl: lvm[1458]: dm-0 (openeuler-root) is created successfully [ 25.571484] device-mapper: ioctl: lvm[1466]: dm-1 (openeuler-swap) is created successfully [ 26.002608] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 26.596890] systemd-journald[875]: Received SIGTERM from PID 1 (systemd). [ 27.204923] SELinux: Runtime disable is deprecated, use selinux=0 on the kernel cmdline. [ 27.223313] SELinux: Disabled at runtime. [ 27.303678] audit: type=1404 audit(1709631975.452:2): enforcing=0 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=0 old-enabled=1 lsm=selinux res=1 [ 27.394966] systemd[1]: systemd v249-63.oe2203sp3 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA -SMACK +SECCOMP +GCRYPT +GNUTLS -OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB -ZSTD +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=legacy) [ 27.458846] systemd[1]: Detected architecture arm64. [ 27.626987] systemd-rc-local-generator[1596]: /etc/rc.d/rc.local is not marked executable, skipping. [ 27.722142] systemd-sysv-generator[1599]: SysV service '/etc/rc.d/init.d/lsvcgss' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [ 27.775931] systemd-sysv-generator[1599]: SysV service '/etc/rc.d/init.d/lustre' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [ 27.831908] systemd-sysv-generator[1599]: SysV service '/etc/rc.d/init.d/mst' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [ 28.457197] systemd[1]: /usr/lib/systemd/system/dbus.socket:5: ListenStream= references a path below legacy directory /var/run/, updating /var/run/dbus/system_bus_socket \xe2\x86\x92 /run/dbus/system_bus_socket; please update the unit file accordingly. [ 28.601618] systemd[1]: /usr/lib/systemd/system/libstoragemgmt.service:7: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether. [ 28.792760] systemd[1]: initrd-switch-root.service: Deactivated successfully. [ 28.817952] systemd[1]: Stopped Switch Root. [ 28.840218] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 28.867383] systemd[1]: Created slice Slice /system/getty. [ 28.891385] systemd[1]: Created slice Slice /system/modprobe. [ 28.916058] systemd[1]: Created slice Slice /system/serial-getty. [ 28.940889] systemd[1]: Created slice Slice /system/sshd-keygen. [ 28.965599] systemd[1]: Created slice Slice /system/systemd-fsck. [ 28.990055] systemd[1]: Created slice User and Session Slice. [ 29.013743] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ 29.047847] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ 29.081535] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. [ 29.116816] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mon@.service instances at once. [ 29.152827] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mds@.service instances at once. [ 29.189027] systemd[1]: Reached target ceph target allowing to start/stop all ceph-mgr@.service instances at once. [ 29.225426] systemd[1]: Reached target ceph target allowing to start/stop all ceph-osd@.service instances at once. [ 29.262017] systemd[1]: Reached target Local Encrypted Volumes. [ 29.285742] systemd[1]: Stopped target Switch Root. [ 29.308037] systemd[1]: Stopped target Initrd File Systems. [ 29.330952] systemd[1]: Stopped target Initrd Root File System. [ 29.354271] systemd[1]: Reached target Path Units. [ 29.376151] systemd[1]: Reached target Remote File Systems. [ 29.398798] systemd[1]: Reached target Slice Units. [ 29.420572] systemd[1]: Reached target Local Verity Integrity Protected Volumes. [ 29.452980] systemd[1]: Listening on Device-mapper event daemon FIFOs. [ 29.484287] systemd[1]: Listening on LVM2 poll daemon socket. [ 29.507979] systemd[1]: Listening on Process Core Dump Socket. [ 29.530589] systemd[1]: Listening on initctl Compatibility Named Pipe. [ 29.561800] systemd[1]: Listening on udev Control Socket. [ 29.583643] systemd[1]: Listening on udev Kernel Socket. [ 29.606098] systemd[1]: Activating swap /dev/mapper/openeuler-swap... [ 29.630072] systemd[1]: Mounting Huge Pages File System... [ 29.644176] VFS: Open a write opened block device exclusively dm-1. current [1605 swapon]. parent [1 systemd] [ 29.671136] Adding 4194300k swap on /dev/mapper/openeuler-swap. Priority:-2 extents:1 across:4194300k FS [ 29.705466] systemd[1]: Mounting POSIX Message Queue File System... [ 29.729386] systemd[1]: Mounting Kernel Debug File System... [ 29.752867] systemd[1]: Mounting Kernel Trace File System... [ 29.794709] systemd[1]: Starting Create List of Static Device Nodes... [ 29.826620] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... [ 29.862820] systemd[1]: Starting Load Kernel Module configfs... [ 29.886498] systemd[1]: Starting Load Kernel Module drm... [ 29.909538] systemd[1]: Starting Load Kernel Module fuse... [ 29.934796] systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped. [ 29.960513] systemd[1]: Stopped Journal Service. [ 29.983291] systemd[1]: Starting Journal Service... [ 29.999794] fuse: init (API version 7.33) [ 30.019222] systemd[1]: Starting Load Kernel Modules... [ 30.041505] systemd[1]: Starting Remount Root and Kernel File Systems... [ 30.073184] systemd[1]: Starting Coldplug All udev Devices... [ 30.089263] EXT4-fs (dm-0): re-mounted. Opts: (null) [ 30.111803] systemd[1]: Activated swap /dev/mapper/openeuler-swap. [ 30.134362] systemd[1]: Started Journal Service. [ 30.307509] systemd-journald[1629]: Received client request to flush runtime journal. [ 30.887312] IPMI message handler: version 39.2 [ 30.909424] ipmi device interface [ 30.918613] hibmc-drm 0000:06:00.0: Adding to iommu group 47 [ 30.930627] checking generic (e4000000 12c000) vs hw (e4000000 2000000) [ 30.930630] fb0: switching to hibmcdrmfb from EFI VGA [ 30.930794] ipmi_si: IPMI System Interface driver [ 30.930822] sd 0:3:111:0: Attached scsi generic sg0 type 0 [ 30.963854] Console: switching to colour dummy device 80x25 [ 30.964244] ipmi_si IPI0001:00: ipmi_platform: probing via ACPI [ 30.976752] ipmi_si IPI0001:00: ipmi_platform: [mem 0x3f00000e4-0x3f00000e7] regsize 1 spacing 1 irq 23 [ 30.976941] [TTM] Zone kernel: Available graphics memory: 246871860 KiB [ 30.994182] [TTM] Zone dma32: Available graphics memory: 2097152 KiB [ 30.994184] [TTM] Initializing pool allocator [ 30.994191] [TTM] Initializing DMA pool allocator [ 30.994273] [drm] forcing VGA-1 connector on [ 31.012289] [drm] Initialized hibmc 1.0.0 20160828 for 0000:06:00.0 on minor 0 [ 31.038016] ipmi_si: Adding ACPI-specified bt state machine [ 31.038154] ipmi_si: Trying ACPI-specified bt state machine at mem address 0x3f00000e4, slave address 0x0, irq 23 [ 31.043747] ipmi_si IPI0001:00: bt cap response too short: 3 [ 31.043749] ipmi_si IPI0001:00: using default values [ 31.043751] ipmi_si IPI0001:00: req2rsp=5 secs retries=2 [ 31.059557] Console: switching to colour frame buffer device 80x30 [ 31.119770] hibmc-drm 0000:06:00.0: [drm] fb0: hibmcdrmfb frame buffer device [ 31.123681] ipmi_si IPI0001:00: The BMC does not support setting the recv irq bit, compensating, but the BMC needs to be fixed. [ 31.151744] ipmi_si IPI0001:00: Using irq 23 [ 31.199775] ipmi_si IPI0001:00: IPMI message handler: Found new BMC (man_id: 0x0007db, prod_id: 0x0001, dev_id: 0x01) [ 31.336805] mlx5_core 0000:81:00.0: MLX5E: StrdRq(0) RqSz(1024) StrdSz(256) RxCqeCmprss(0 basic) [ 31.347332] mlx5_core 0000:81:00.0: MLX5E: StrdRq(0) RqSz(1024) StrdSz(256) RxCqeCmprss(0 basic) [ 31.427012] mlx5_core 0000:81:00.0 ibp129s0f0: renamed from ib0 [ 31.427587] mlx5_core 0000:81:00.1: MLX5E: StrdRq(0) RqSz(1024) StrdSz(256) RxCqeCmprss(0 basic) [ 31.431954] ipmi_si IPI0001:00: IPMI bt interface initialized [ 31.452210] mlx5_core 0000:81:00.1: MLX5E: StrdRq(0) RqSz(1024) StrdSz(256) RxCqeCmprss(0 basic) [ 31.453639] ipmi_ssif: IPMI SSIF Interface driver [ 31.539235] mlx5_core 0000:81:00.1 ibp129s0f1: renamed from ib0 [ 31.541493] xprtrdma: xprtrdma is obsoleted, loading rpcrdma instead [ 31.553050] svcrdma: svcrdma is obsoleted, loading rpcrdma instead [ 31.718643] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null) [ 32.019261] device-mapper: ioctl: lvm[2731]: dm-2 (openeuler-home) is created successfully [ 32.597760] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: (null) [ 33.858536] VFIO - User Level meta-driver version: 0.3 [ 34.107379] hns3 0000:7d:00.0 enp125s0f0: link up [ 34.114767] IPv6: ADDRCONF(NETDEV_CHANGE): enp125s0f0: link becomes ready [ 34.128887] hns3 0000:7d:00.2 enp125s0f2: link up [ 34.141837] hns3 0000:bd:00.0 enp189s0f0: link up [ 34.152843] hns3 0000:bd:00.2 enp189s0f2: link up [ 34.158579] hns3 0000:bd:00.3 enp189s0f3: link up [ 34.435460] ixgbe 0000:01:00.0: registered PHC device on enp1s0f0 [ 34.820861] ixgbe 0000:01:00.1: registered PHC device on enp1s0f1 [ 35.277475] IPv6: ADDRCONF(NETDEV_CHANGE): enp125s0f2: link becomes ready [ 35.286497] IPv6: ADDRCONF(NETDEV_CHANGE): enp189s0f0: link becomes ready [ 35.295142] IPv6: ADDRCONF(NETDEV_CHANGE): enp189s0f2: link becomes ready [ 35.303795] IPv6: ADDRCONF(NETDEV_CHANGE): enp189s0f3: link becomes ready [ 36.717536] IPv6: ADDRCONF(NETDEV_CHANGE): ibp129s0f1: link becomes ready [ 37.240932] IPv6: ADDRCONF(NETDEV_CHANGE): ibp129s0f0: link becomes ready [ 2694.764249] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [ 2694.773025] alg: No test for adler32 (adler32-zlib) [ 2695.521720] Key type ._llcrypt registered [ 2695.526450] Key type .llcrypt registered [ 2695.544210] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 2696.087908] RPC: Registered named UNIX socket transport module. [ 2696.094579] RPC: Registered udp transport module. [ 2696.100001] RPC: Registered tcp transport module. [ 2696.105424] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 2696.144421] lnet: unknown parameter '#' ignored [ 2696.149664] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 2696.156311] lnet: unknown parameter '#' ignored [ 2696.161566] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 2696.289241] Lustre: Lustre: Build Version: 2.15.4 [ 2696.396242] LNet: Using FastReg for registration [ 2696.604872] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [ 2698.157561] Key type lgssc registered [ 2698.382629] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3275.039027] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3292.790074] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 3312.218955] device-mapper: ioctl: dmsetup[8010]: dm-3 (ost1_flakey) is created successfully [ 3313.432060] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 3313.554945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3313.567432] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 3314.665697] Lustre: lustre-OST0000: new disk, initializing [ 3314.676245] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 3314.698900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3314.718813] VFS: Open an exclusive opened block device for write dm-3. current [8394 tune2fs]. parent [8393 sh] [ 3315.545384] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [ 3320.641921] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3329.432161] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3355.571897] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 3370.931761] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 3372.979521] Lustre: 5247:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709635313/real 1709635313] req@00000000d7d634de x1792681856472576/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709635320 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [ 3373.013112] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [ 3385.267541] LustreError: 9298:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [ 3385.319200] Lustre: server umount lustre-OST0000 complete [ 3386.782782] device-mapper: ioctl: dmsetup[9660]: dm-3 (ost1_flakey) is removed successfully [ 3399.280433] Lustre: DEBUG MARKER: server5: executing unload_modules_local [ 3400.008358] Key type lgssc unregistered [ 3400.163375] LNet: 10231:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3401.331265] LNet: Removed LNI 192.168.0.85@o2ib [ 3401.691221] Key type .llcrypt unregistered [ 3401.705768] Key type ._llcrypt unregistered [ 3414.713460] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [ 3414.723310] alg: No test for adler32 (adler32-zlib) [ 3415.471251] Key type ._llcrypt registered [ 3415.476885] Key type .llcrypt registered [ 3415.499414] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 3415.883183] lnet: unknown parameter '#' ignored [ 3415.889210] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 3415.896238] lnet: unknown parameter '#' ignored [ 3415.901853] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 3415.976208] Lustre: Lustre: Build Version: 2.15.4 [ 3416.035262] LNet: Using FastReg for registration [ 3416.239197] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [ 3417.787106] Key type lgssc registered [ 3417.947438] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4003.968016] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4021.059756] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 4040.486675] device-mapper: ioctl: dmsetup[13057]: dm-3 (ost1_flakey) is created successfully [ 4041.690386] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4041.782796] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4041.795182] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 4042.867495] Lustre: lustre-OST0000: new disk, initializing [ 4042.874510] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 4042.896416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4042.916273] VFS: Open an exclusive opened block device for write dm-3. current [13441 tune2fs]. parent [13440 sh] [ 4043.643541] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4043.746257] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [ 4057.469562] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4078.765452] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 4095.149034] Lustre: 11482:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709636036/real 1709636036] req@000000009111e870 x1792682610398528/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709636043 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:3.0' [ 4095.182465] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [ 4095.198765] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 4109.229058] LustreError: 14343:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [ 4109.278206] Lustre: server umount lustre-OST0000 complete [ 4110.760222] device-mapper: ioctl: dmsetup[14706]: dm-3 (ost1_flakey) is removed successfully [ 4123.195898] Lustre: DEBUG MARKER: server5: executing unload_modules_local [ 4123.982166] Key type lgssc unregistered [ 4124.192917] LNet: 15277:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4125.388787] LNet: Removed LNI 192.168.0.85@o2ib [ 4125.760811] Key type .llcrypt unregistered [ 4125.775379] Key type ._llcrypt unregistered [ 4138.631460] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [ 4138.641618] alg: No test for adler32 (adler32-zlib) [ 4139.388787] Key type ._llcrypt registered [ 4139.394433] Key type .llcrypt registered [ 4139.420052] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 4139.805628] lnet: unknown parameter '#' ignored [ 4139.811658] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 4139.818685] lnet: unknown parameter '#' ignored [ 4139.824304] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 4139.897325] Lustre: Lustre: Build Version: 2.15.4 [ 4139.955420] LNet: Using FastReg for registration [ 4140.159521] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [ 4141.708611] Key type lgssc registered [ 4141.856869] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4727.796497] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4744.867049] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 4764.164352] device-mapper: ioctl: dmsetup[18082]: dm-3 (ost1_flakey) is created successfully [ 4765.371642] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 4765.467654] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4765.480424] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 4766.556873] Lustre: lustre-OST0000: new disk, initializing [ 4766.567214] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 4766.588291] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4766.605779] VFS: Open an exclusive opened block device for write dm-3. current [18466 tune2fs]. parent [18465 sh] [ 4767.410557] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [ 4768.309160] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4781.118465] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4807.590969] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 4824.998508] Lustre: 16564:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709636765/real 1709636765] req@000000007e57e9f1 x1792683369567744/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709636772 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [ 4825.031870] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [ 4825.048185] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 4846.246342] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 2. Is it stuck? [ 4846.262468] LustreError: 19372:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [ 4846.310729] Lustre: server umount lustre-OST0000 complete [ 4847.794761] device-mapper: ioctl: dmsetup[19735]: dm-3 (ost1_flakey) is removed successfully [ 4860.275419] Lustre: DEBUG MARKER: server5: executing unload_modules_local [ 4861.026638] Key type lgssc unregistered [ 4861.290272] LNet: 20306:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4862.470182] LNet: Removed LNI 192.168.0.85@o2ib [ 4862.806392] Key type .llcrypt unregistered [ 4862.820584] Key type ._llcrypt unregistered [ 4875.780973] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [ 4875.791105] alg: No test for adler32 (adler32-zlib) [ 4876.538141] Key type ._llcrypt registered [ 4876.543286] Key type .llcrypt registered [ 4876.566000] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 4876.952341] lnet: unknown parameter '#' ignored [ 4876.958116] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 4876.965166] lnet: unknown parameter '#' ignored [ 4876.970806] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 4877.044820] Lustre: Lustre: Build Version: 2.15.4 [ 4877.106536] LNet: Using FastReg for registration [ 4877.313961] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [ 4878.858014] Key type lgssc registered [ 4879.025384] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5465.227200] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5482.172581] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 5501.519808] device-mapper: ioctl: dmsetup[23122]: dm-3 (ost1_flakey) is created successfully [ 5502.742606] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 5502.837046] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5502.849761] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 5503.922125] Lustre: lustre-OST0000: new disk, initializing [ 5503.929337] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 5503.950915] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5503.970873] VFS: Open an exclusive opened block device for write dm-3. current [23505 tune2fs]. parent [23504 sh] [ 5504.558535] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5504.796321] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [ 5518.544884] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5544.864148] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 5556.127906] Lustre: 21567:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709637497/real 1709637497] req@00000000345217b6 x1792684143416640/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709637504 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [ 5556.161341] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [ 5556.177735] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 5570.467926] LustreError: 24408:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [ 5570.516566] Lustre: server umount lustre-OST0000 complete [ 5572.000615] device-mapper: ioctl: dmsetup[24770]: dm-3 (ost1_flakey) is removed successfully [ 5592.658435] Lustre: DEBUG MARKER: server5: executing unload_modules_local [ 5593.411017] Key type lgssc unregistered [ 5593.607681] LNet: 25340:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5594.783564] LNet: Removed LNI 192.168.0.85@o2ib [ 5595.155749] Key type .llcrypt unregistered [ 5595.170325] Key type ._llcrypt unregistered [ 5608.335118] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [ 5608.344673] alg: No test for adler32 (adler32-zlib) [ 5609.091551] Key type ._llcrypt registered [ 5609.096926] Key type .llcrypt registered [ 5609.119690] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 5609.516590] lnet: unknown parameter '#' ignored [ 5609.522368] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 5609.529419] lnet: unknown parameter '#' ignored [ 5609.535063] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 5609.609674] Lustre: Lustre: Build Version: 2.15.4 [ 5609.670875] LNet: Using FastReg for registration [ 5609.881826] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [ 5611.431402] Key type lgssc registered [ 5611.582892] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6197.527316] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6214.615111] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 6234.064925] device-mapper: ioctl: dmsetup[28159]: dm-3 (ost1_flakey) is created successfully [ 6235.269043] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6235.358675] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6235.371088] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 6236.189607] Lustre: lustre-OST0000: Not available for connect from 192.168.0.84@o2ib (not set up) [ 6236.443574] Lustre: lustre-OST0000: new disk, initializing [ 6236.450779] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 6236.472236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6236.489508] VFS: Open an exclusive opened block device for write dm-3. current [28544 tune2fs]. parent [28543 sh] [ 6237.308098] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [ 6241.316597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6250.986718] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6272.409629] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 6289.817298] Lustre: 26656:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709638230/real 1709638230] req@000000001ed7efbb x1792684910974208/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709638237 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [ 6289.850661] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [ 6289.866916] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 6302.873303] LustreError: 29460:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [ 6302.922364] Lustre: server umount lustre-OST0000 complete [ 6304.391820] device-mapper: ioctl: dmsetup[29821]: dm-3 (ost1_flakey) is removed successfully [ 6316.894239] Lustre: DEBUG MARKER: server5: executing unload_modules_local [ 6317.625809] Key type lgssc unregistered [ 6317.801122] LNet: 30391:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6319.001045] LNet: Removed LNI 192.168.0.85@o2ib [ 6319.325167] Key type .llcrypt unregistered [ 6319.339737] Key type ._llcrypt unregistered [ 6332.270024] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [ 6332.279957] alg: No test for adler32 (adler32-zlib) [ 6333.029017] Key type ._llcrypt registered [ 6333.034640] Key type .llcrypt registered [ 6333.058365] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 6333.445491] lnet: unknown parameter '#' ignored [ 6333.451456] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 6333.458487] lnet: unknown parameter '#' ignored [ 6333.464101] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 6333.536973] Lustre: Lustre: Build Version: 2.15.4 [ 6333.596993] LNet: Using FastReg for registration [ 6333.800250] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [ 6335.356870] Key type lgssc registered [ 6335.509167] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6921.540227] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6938.721816] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 6958.118332] device-mapper: ioctl: dmsetup[33175]: dm-3 (ost1_flakey) is created successfully [ 6959.323174] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 6959.420517] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6959.432937] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 6960.227062] Lustre: lustre-OST0000: Not available for connect from 192.168.0.83@o2ib (not set up) [ 6960.509009] Lustre: lustre-OST0000: new disk, initializing [ 6960.516075] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 6960.538915] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6960.558146] VFS: Open an exclusive opened block device for write dm-3. current [33559 tune2fs]. parent [33558 sh] [ 6960.641035] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6961.379486] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [ 6975.099075] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7001.491022] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 7013.778732] Lustre: 31618:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709638954/real 1709638954] req@00000000fe32d842 x1792685670143232/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709638961 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [ 7013.812173] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [ 7018.898963] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 7038.354502] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 2. Is it stuck? [ 7038.370597] LustreError: 34461:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [ 7038.420250] Lustre: server umount lustre-OST0000 complete [ 7039.876916] device-mapper: ioctl: dmsetup[34824]: dm-3 (ost1_flakey) is removed successfully [ 7052.313458] Lustre: DEBUG MARKER: server5: executing unload_modules_local [ 7053.089457] Key type lgssc unregistered [ 7053.266463] LNet: 35394:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7054.450369] LNet: Removed LNI 192.168.0.85@o2ib [ 7054.786630] Key type .llcrypt unregistered [ 7054.800917] Key type ._llcrypt unregistered [ 7067.930059] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [ 7067.939796] alg: No test for adler32 (adler32-zlib) [ 7068.686328] Key type ._llcrypt registered [ 7068.691488] Key type .llcrypt registered [ 7068.714621] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 7069.099581] lnet: unknown parameter '#' ignored [ 7069.105323] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 7069.112364] lnet: unknown parameter '#' ignored [ 7069.117987] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [ 7069.193567] Lustre: Lustre: Build Version: 2.15.4 [ 7069.252539] LNet: Using FastReg for registration [ 7069.466046] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [ 7071.014194] Key type lgssc registered [ 7071.168513] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7657.049425] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7673.941581] Lustre: DEBUG MARKER: server5: executing load_modules_local [ 7693.419786] device-mapper: ioctl: dmsetup[38204]: dm-3 (ost1_flakey) is created successfully [ 7694.627770] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [ 7694.725035] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7694.737385] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [ 7695.773632] Lustre: lustre-OST0000: Not available for connect from 192.168.0.83@o2ib (not set up) [ 7695.810580] Lustre: lustre-OST0000: new disk, initializing [ 7695.817631] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [ 7695.838918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7695.858916] VFS: Open an exclusive opened block device for write dm-3. current [38588 tune2fs]. parent [38587 sh] [ 7696.682316] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [ 7697.690351] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7710.222606] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7736.716322] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 7749.003978] Lustre: 36650:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709639689/real 1709639689] req@000000005bdcb611 x1792686441895232/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709639696 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [ 7749.037410] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [ 7749.053747] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [ 7762.828006] LustreError: 39492:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [ 7762.876548] Lustre: server umount lustre-OST0000 complete [ 7764.373548] device-mapper: ioctl: dmsetup[39853]: dm-3 (ost1_flakey) is removed successfully [ 7776.796091] Lustre: DEBUG MARKER: server5: executing unload_modules_local [ 7777.552550] Key type lgssc unregistered [ 7777.755818] LNet: 40421:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7778.955711] LNet: Removed LNI 192.168.0.85@o2ib [ 7779.259806] Key type .llcrypt unregistered [ 7779.274080] Key type ._llcrypt unregistered [55978.726583] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [55978.736536] alg: No test for adler32 (adler32-zlib) [55979.483622] Key type ._llcrypt registered [55979.489267] Key type .llcrypt registered [55979.511812] Lustre: DEBUG MARKER: server5: executing unload_modules_local [55979.553483] Key type .llcrypt unregistered [55979.567542] Key type ._llcrypt unregistered [56008.012183] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [56008.021854] alg: No test for adler32 (adler32-zlib) [56008.771327] Key type ._llcrypt registered [56008.776436] Key type .llcrypt registered [56008.799029] Lustre: DEBUG MARKER: server5: executing load_modules_local [56009.166651] lnet: unknown parameter '#' ignored [56009.172364] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [56009.179356] lnet: unknown parameter '#' ignored [56009.184940] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [56009.259424] Lustre: Lustre: Build Version: 2.15.4 [56009.317567] LNet: Using FastReg for registration [56009.529223] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [56011.079184] Key type lgssc registered [56011.235370] Lustre: Echo OBD driver; http://www.lustre.org/ [56276.629658] Lustre: DEBUG MARKER: server5: executing unload_modules_local [56277.113683] Key type lgssc unregistered [56277.240905] LNet: 44640:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [56278.420799] LNet: Removed LNI 192.168.0.85@o2ib [56278.669096] Key type .llcrypt unregistered [56278.683155] Key type ._llcrypt unregistered [56293.134640] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [56293.144130] alg: No test for adler32 (adler32-zlib) [56293.892753] Key type ._llcrypt registered [56293.897944] Key type .llcrypt registered [56293.924100] Lustre: DEBUG MARKER: server5: executing load_modules_local [56294.325815] lnet: unknown parameter '#' ignored [56294.331704] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [56294.338673] lnet: unknown parameter '#' ignored [56294.344228] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [56294.419202] Lustre: Lustre: Build Version: 2.15.4 [56294.477597] LNet: Using FastReg for registration [56294.688834] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [56296.232607] Key type lgssc registered [56296.389553] Lustre: Echo OBD driver; http://www.lustre.org/ [56882.293419] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [56899.252450] Lustre: DEBUG MARKER: server5: executing load_modules_local [56918.700065] device-mapper: ioctl: dmsetup[47430]: dm-3 (ost1_flakey) is created successfully [56919.905706] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [56920.004748] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [56920.017274] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [56920.292387] Lustre: lustre-OST0000: Not available for connect from 192.168.0.84@o2ib (not set up) [56921.088652] Lustre: lustre-OST0000: new disk, initializing [56921.095494] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [56921.116505] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [56921.133037] VFS: Open an exclusive opened block device for write dm-3. current [47815 tune2fs]. parent [47814 sh] [56921.954028] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [56925.531387] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [56935.490896] Lustre: DEBUG MARKER: Using TIMEOUT=20 [63745.937621] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [63758.225347] Lustre: 45879:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709695699/real 1709695699] req@000000002ef3999b x1792738108193600/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709695706 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:3.0' [63758.258794] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [63761.041314] Lustre: ll_ost01_003: service thread pid 47806 was inactive for 41.588 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [63761.041317] Pid: 48908, comm: ll_ost01_005 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [63761.041320] Call Trace TBD: [63761.041322] Lustre: ll_ost01_001: service thread pid 47757 was inactive for 41.588 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [63761.062512] Lustre: Skipped 2 previous similar messages [63761.074706] Pid: 48909, comm: ll_ost01_006 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [63761.074709] Call Trace TBD: [63761.118397] Pid: 47806, comm: ll_ost01_003 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [63761.130545] Call Trace TBD: [63761.297518] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [63776.145399] LustreError: 49127:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [63776.739035] Lustre: server umount lustre-OST0000 complete [63779.005543] device-mapper: ioctl: dmsetup[49489]: dm-3 (ost1_flakey) is removed successfully [63794.127771] Lustre: DEBUG MARKER: server5: executing unload_modules_local [63794.890007] Key type lgssc unregistered [63795.061125] LNet: 50061:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [63796.241026] LNet: Removed LNI 192.168.0.85@o2ib [63796.565070] Key type .llcrypt unregistered [63796.579294] Key type ._llcrypt unregistered [63809.528377] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [63809.538218] alg: No test for adler32 (adler32-zlib) [63810.285001] Key type ._llcrypt registered [63810.290334] Key type .llcrypt registered [63810.314169] Lustre: DEBUG MARKER: server5: executing load_modules_local [63810.710536] lnet: unknown parameter '#' ignored [63810.716308] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [63810.723374] lnet: unknown parameter '#' ignored [63810.729007] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [63810.805671] Lustre: Lustre: Build Version: 2.15.4 [63810.865899] LNet: Using FastReg for registration [63811.070706] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [63812.616849] Key type lgssc registered [63812.772150] Lustre: Echo OBD driver; http://www.lustre.org/ [64399.574362] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [64416.981461] Lustre: DEBUG MARKER: server5: executing load_modules_local [64436.430772] device-mapper: ioctl: dmsetup[52858]: dm-3 (ost1_flakey) is created successfully [64437.643487] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [64437.740611] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [64437.752918] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [64438.824857] Lustre: lustre-OST0000: new disk, initializing [64438.831713] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [64438.854249] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [64438.871697] VFS: Open an exclusive opened block device for write dm-3. current [53242 tune2fs]. parent [53241 sh] [64439.679776] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [64439.802669] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [64453.249518] Lustre: DEBUG MARKER: Using TIMEOUT=20 [72282.692685] Lustre: ll_ost01_010: service thread pid 54129 was inactive for 41.335 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [72282.692687] Pid: 54128, comm: ll_ost01_009 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [72282.692688] Lustre: ll_ost01_000: service thread pid 53184 was inactive for 41.336 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [72282.692692] Call Trace TBD: [72282.692693] Lustre: Skipped 4 previous similar messages [72282.692827] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [72282.713913] Lustre: Skipped 2 previous similar messages [72282.726039] Pid: 54124, comm: ll_ost01_005 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [72282.790706] Call Trace TBD: [72282.794676] Pid: 54129, comm: ll_ost01_010 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [72282.806770] Call Trace TBD: [72284.355934] Lustre: lustre-OST0000: haven't heard from client 3211ee2f-4536-4f1c-b43e-87c8e8f1a6d8 (at 192.168.0.82@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 0000000023c8fd97, cur 1709704232 expire 1709704202 last 1709704185 [72284.381076] LustreError: 54124:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000000039b748 x1792746040806528/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [72284.406134] LustreError: 54124:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 3 previous similar messages [72285.663922] LustreError: 54123:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000128abe05 x1792746040806976/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [72285.688722] LustreError: 54123:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 4 previous similar messages [72286.867336] LustreError: 54128:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000007a4dc11e x1792746040807360/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [72286.892220] LustreError: 54128:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 5 previous similar messages [72288.876941] LustreError: 54124:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000000039b748 x1792746040807616/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [72294.980595] Lustre: 51312:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709704236/real 1709704236] req@000000001ad782b8 x1792746040807424/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709704243 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [72295.014272] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [72295.030662] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [72306.244751] LustreError: 54354:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [72307.226172] Lustre: server umount lustre-OST0000 complete [72309.843568] device-mapper: ioctl: dmsetup[54719]: dm-3 (ost1_flakey) is removed successfully [72321.495060] Lustre: DEBUG MARKER: server5: executing unload_modules_local [72322.345107] Key type lgssc unregistered [72322.560429] LNet: 55290:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [72323.748343] LNet: Removed LNI 192.168.0.85@o2ib [72324.120612] Key type .llcrypt unregistered [72324.134817] Key type ._llcrypt unregistered [72336.972267] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [72336.982124] alg: No test for adler32 (adler32-zlib) [72337.732340] Key type ._llcrypt registered [72337.737585] Key type .llcrypt registered [72337.760243] Lustre: DEBUG MARKER: server5: executing load_modules_local [72338.155052] lnet: unknown parameter '#' ignored [72338.160840] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [72338.167890] lnet: unknown parameter '#' ignored [72338.173538] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [72338.248438] Lustre: Lustre: Build Version: 2.15.4 [72338.310384] LNet: Using FastReg for registration [72338.516078] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [72340.064173] Key type lgssc registered [72340.221042] Lustre: Echo OBD driver; http://www.lustre.org/ [72927.840259] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [72945.595813] Lustre: DEBUG MARKER: server5: executing load_modules_local [72965.069155] device-mapper: ioctl: dmsetup[58115]: dm-3 (ost1_flakey) is created successfully [72966.282664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [72966.372117] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [72966.384522] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [72967.460437] Lustre: lustre-OST0000: new disk, initializing [72967.467517] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [72967.489062] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [72967.509000] VFS: Open an exclusive opened block device for write dm-3. current [58499 tune2fs]. parent [58498 sh] [72968.337125] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [72972.494153] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [72981.885299] Lustre: DEBUG MARKER: Using TIMEOUT=20 [75713.363675] Lustre: lustre-OST0000: Client 933d0673-13c1-4fa4-8175-de980cecf2f6 (at 192.168.0.82@o2ib) reconnecting [81526.128856] Lustre: lustre-OST0000: haven't heard from client 8f2af079-bf7b-454b-b66b-bfa504cc2e27 (at 192.168.0.81@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 00000000088ea2cd, cur 1709713474 expire 1709713444 last 1709713427 [81526.153837] LustreError: 58440:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000eb5f287a x1792755032358656/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [81526.178510] LustreError: 58440:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 4 previous similar messages [81528.770536] LustreError: 59327:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000004f3dba31 x1792755032359360/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [81528.795367] LustreError: 59327:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 3 previous similar messages [81530.502835] LustreError: 59047:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000c5169a1a x1792755032359552/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [81530.527717] LustreError: 59047:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 2 previous similar messages [81533.696148] LustreError: 59323:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000c703cfc3 x1792755032359808/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [81548.530101] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [81560.817781] Lustre: 56557:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709713502/real 1709713502] req@000000006491a880 x1792755032360576/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709713509 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [81560.851527] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [81572.849679] Lustre: 61257:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709713515/real 1709713515] req@000000001afbecf8 x1792755032361344/t0(0) o39->lustre-MDT0001-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709713521 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [81583.182505] Lustre: server umount lustre-OST0000 complete [81587.505470] device-mapper: ioctl: dmsetup[61622]: dm-3 (ost1_flakey) is removed successfully [81607.469262] Lustre: DEBUG MARKER: server5: executing unload_modules_local [81608.258301] Key type lgssc unregistered [81608.421462] LNet: 62194:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [81609.617336] LNet: Removed LNI 192.168.0.85@o2ib [81609.981538] Key type .llcrypt unregistered [81609.996133] Key type ._llcrypt unregistered [81622.965048] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [81622.975201] alg: No test for adler32 (adler32-zlib) [81623.725356] Key type ._llcrypt registered [81623.731039] Key type .llcrypt registered [81623.753893] Lustre: DEBUG MARKER: server5: executing load_modules_local [81624.151020] lnet: unknown parameter '#' ignored [81624.157055] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [81624.164092] lnet: unknown parameter '#' ignored [81624.169710] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [81624.242850] Lustre: Lustre: Build Version: 2.15.4 [81624.301659] LNet: Using FastReg for registration [81624.514493] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [81626.061177] Key type lgssc registered [81626.222680] Lustre: Echo OBD driver; http://www.lustre.org/ [82215.069432] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [82233.340974] Lustre: DEBUG MARKER: server5: executing load_modules_local [82252.867611] device-mapper: ioctl: dmsetup[65022]: dm-3 (ost1_flakey) is created successfully [82254.069444] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [82254.170091] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [82254.182497] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [82255.024342] Lustre: lustre-OST0000: Not available for connect from 192.168.0.84@o2ib (not set up) [82255.257748] Lustre: lustre-OST0000: new disk, initializing [82255.264886] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [82255.286272] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [82255.305924] VFS: Open an exclusive opened block device for write dm-3. current [65407 tune2fs]. parent [65406 sh] [82256.125305] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [82260.087479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [82269.779516] Lustre: DEBUG MARKER: Using TIMEOUT=20 [90507.552044] Lustre: lustre-OST0000: haven't heard from client bbfa3c5b-fcf7-43cd-af67-b136c6da6ca4 (at 192.168.0.81@o2ib) in 50 seconds. I think it's dead, and I am evicting it. exp 00000000cc42478d, cur 1709722456 expire 1709722426 last 1709722406 [90534.305539] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [90546.593272] Lustre: 63497:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709722488/real 1709722488] req@00000000baeb92c3 x1792764749938752/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709722495 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:1.0' [90546.626977] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [90556.065201] Lustre: 66537:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709722498/real 1709722498] req@000000008047e3d5 x1792764749939392/t0(0) o39->lustre-MDT0001-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709722504 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [90558.945371] LustreError: 66537:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [90562.978681] Lustre: server umount lustre-OST0000 complete [90570.052811] device-mapper: ioctl: dmsetup[66905]: dm-3 (ost1_flakey) is removed successfully [90587.656999] Lustre: DEBUG MARKER: server5: executing unload_modules_local [90588.381661] Key type lgssc unregistered [90588.580999] LNet: 67478:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [90589.760897] LNet: Removed LNI 192.168.0.85@o2ib [90590.124959] Key type .llcrypt unregistered [90590.139171] Key type ._llcrypt unregistered [90603.140368] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [90603.150546] alg: No test for adler32 (adler32-zlib) [90603.900876] Key type ._llcrypt registered [90603.906553] Key type .llcrypt registered [90603.933087] Lustre: DEBUG MARKER: server5: executing load_modules_local [90604.349552] lnet: unknown parameter '#' ignored [90604.355536] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [90604.362561] lnet: unknown parameter '#' ignored [90604.368178] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [90604.441109] Lustre: Lustre: Build Version: 2.15.4 [90604.501194] LNet: Using FastReg for registration [90604.709983] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [90606.260727] Key type lgssc registered [90606.415808] Lustre: Echo OBD driver; http://www.lustre.org/ [91193.928433] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [91214.666865] Lustre: DEBUG MARKER: server5: executing load_modules_local [91234.105781] device-mapper: ioctl: dmsetup[70315]: dm-3 (ost1_flakey) is created successfully [91235.319964] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [91235.416359] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [91235.429130] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [91236.504830] Lustre: lustre-OST0000: new disk, initializing [91236.511878] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [91236.534813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [91236.553549] VFS: Open an exclusive opened block device for write dm-3. current [70700 tune2fs]. parent [70699 sh] [91237.374163] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [91244.586007] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [91251.004029] Lustre: DEBUG MARKER: Using TIMEOUT=20 [110612.717908] Lustre: ll_ost01_008: service thread pid 71795 was inactive for 41.080 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [110612.717926] Lustre: ll_ost01_001: service thread pid 70643 was inactive for 41.079 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [110612.717928] Pid: 71793, comm: ll_ost01_006 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [110612.717931] Lustre: ll_ost01_009: service thread pid 71796 was inactive for 41.081 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [110612.717932] Call Trace TBD: [110612.717934] Lustre: Skipped 3 previous similar messages [110612.717936] Pid: 71797, comm: ll_ost01_010 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [110612.717937] Call Trace TBD: [110612.739223] Lustre: Skipped 1 previous similar message [110612.818909] Pid: 71795, comm: ll_ost01_008 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [110612.831050] Call Trace TBD: [110614.268698] Lustre: lustre-OST0000: haven't heard from client 608206de-40bc-4a09-8be7-a064a14ac832 (at 192.168.0.82@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 0000000055fd4321, cur 1709742563 expire 1709742533 last 1709742515 [110614.295069] LustreError: 71791:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000028a21e54 x1792774205008960/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [110614.295072] LustreError: 71797:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000e5dd36a5 x1792774205009024/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [110614.345393] LustreError: 71791:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 3 previous similar messages [110618.224822] LustreError: 71793:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000006f0fe44a x1792774205009536/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [110618.249809] LustreError: 71793:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 3 previous similar messages [110619.496441] LustreError: 70644:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000881e5744 x1792774205009600/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [110622.163698] LustreError: 71793:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000006f0fe44a x1792774205009984/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [110622.188834] LustreError: 71793:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 2 previous similar messages [110636.702515] LustreError: 71795:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000006f34592b x1792774205010688/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [110636.727731] LustreError: 71795:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 1 previous similar message [110648.045538] LustreError: 71793:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000481e5f25 x1792774205011328/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [110648.070708] LustreError: 71793:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 3 previous similar messages [110651.373745] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [110658.541504] Lustre: 68783:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709742600/real 1709742600] req@000000003c337b65 x1792774205011456/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709742607 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [110658.575548] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [110670.829408] Lustre: 72022:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709742613/real 1709742613] req@000000006afa5aa3 x1792774205012864/t0(0) o39->lustre-MDT0001-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709742619 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [110676.973526] LustreError: 72022:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [110683.872619] Lustre: server umount lustre-OST0000 complete [110693.061440] device-mapper: ioctl: dmsetup[72387]: dm-3 (ost1_flakey) is removed successfully [110711.667307] Lustre: DEBUG MARKER: server5: executing unload_modules_local [110712.385825] Key type lgssc unregistered [110712.585112] LNet: 72960:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [110713.773009] LNet: Removed LNI 192.168.0.85@o2ib [110714.097034] Key type .llcrypt unregistered [110714.111351] Key type ._llcrypt unregistered [110727.271627] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [110727.281541] alg: No test for adler32 (adler32-zlib) [110728.029002] Key type ._llcrypt registered [110728.034764] Key type .llcrypt registered [110728.058027] Lustre: DEBUG MARKER: server5: executing load_modules_local [110728.443587] lnet: unknown parameter '#' ignored [110728.449675] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [110728.456791] lnet: unknown parameter '#' ignored [110728.462492] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [110728.534670] Lustre: Lustre: Build Version: 2.15.4 [110728.594143] LNet: Using FastReg for registration [110728.808007] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [110730.352857] Key type lgssc registered [110730.506696] Lustre: Echo OBD driver; http://www.lustre.org/ [111320.081037] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [111343.775770] Lustre: DEBUG MARKER: server5: executing load_modules_local [111363.207182] device-mapper: ioctl: dmsetup[75748]: dm-3 (ost1_flakey) is created successfully [111364.415671] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [111364.516696] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [111364.529139] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [111365.605114] Lustre: lustre-OST0000: new disk, initializing [111365.612263] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [111365.633656] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [111365.651067] VFS: Open an exclusive opened block device for write dm-3. current [76132 tune2fs]. parent [76131 sh] [111366.468486] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [111370.100585] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [111380.127279] Lustre: DEBUG MARKER: Using TIMEOUT=20 [144537.522821] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [144537.541251] IPv6: ADDRCONF(NETDEV_CHANGE): enp1s0f1: link becomes ready [144539.509280] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [144811.962860] Lustre: ll_ost01_001: service thread pid 76075 was inactive for 40.824 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [144811.962861] Lustre: ll_ost01_010: service thread pid 77641 was inactive for 40.817 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [144811.962864] Pid: 77636, comm: ll_ost01_005 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [144811.962869] Lustre: Skipped 2 previous similar messages [144811.962870] Call Trace TBD: [144811.962873] Pid: 77635, comm: ll_ost01_004 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [144811.962874] Call Trace TBD: [144811.980089] Lustre: Skipped 2 previous similar messages [144812.046582] Pid: 77641, comm: ll_ost01_010 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [144812.058706] Call Trace TBD: [144812.087964] Lustre: lustre-OST0000: haven't heard from client 52a8b960-64d9-474f-b4d1-69d2677c2777 (at 192.168.0.81@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 00000000e2d39119, cur 1709776761 expire 1709776731 last 1709776713 [144812.113681] LustreError: 77635:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000b889d926 x1792795397580864/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144812.138748] LustreError: 77635:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 4 previous similar messages [144812.705223] LustreError: 77636:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000001b2aafa9 x1792795397581888/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144812.730143] LustreError: 77636:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 11 previous similar messages [144827.719100] LustreError: 77641:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000009a90b760 x1792795397582592/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144827.744150] LustreError: 77641:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 1 previous similar message [144834.168226] LustreError: 77638:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000000b4bb45e x1792795397582848/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144866.568823] LustreError: 77637:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000088dc2e9a x1792795397584320/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144866.594102] LustreError: 77637:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 1 previous similar message [144879.835007] LustreError: 76075:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000068b567de x1792795397584896/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144879.860294] LustreError: 76075:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 2 previous similar messages [144895.873226] LustreError: 77641:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000009a90b760 x1792795397585536/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144913.405941] LustreError: 76067:0:(ldlm_lockd.c:257:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 192.168.0.81@o2ib ns: filter-lustre-OST0000_UUID lock: 00000000f72d375b/0xc8b439a6a222e40e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x57e21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400010020 nid: 192.168.0.81@o2ib remote: 0xacbedd9d91f50f84 expref: 68725 pid: 76829 timeout: 144913 lvb_type: 0 [144929.974641] LustreError: 77641:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000009a90b760 x1792795397588224/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [144929.999963] LustreError: 77641:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 20 previous similar messages [145833.137838] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [145840.305629] Lustre: 74241:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709777782/real 1709777782] req@000000002a10224c x1792795397624000/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709777789 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [145840.339121] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [145840.355492] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [145853.407021] Lustre: server umount lustre-OST0000 complete [145861.093613] device-mapper: ioctl: dmsetup[78262]: dm-3 (ost1_flakey) is removed successfully [145879.597819] Lustre: DEBUG MARKER: server5: executing unload_modules_local [145880.293752] Key type lgssc unregistered [145880.517367] LNet: 78835:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [145881.713259] LNet: Removed LNI 192.168.0.85@o2ib [145882.045340] Key type .llcrypt unregistered [145882.059704] Key type ._llcrypt unregistered [145895.033507] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [145895.043311] alg: No test for adler32 (adler32-zlib) [145895.793271] Key type ._llcrypt registered [145895.799052] Key type .llcrypt registered [145895.822260] Lustre: DEBUG MARKER: server5: executing load_modules_local [145896.234652] lnet: unknown parameter '#' ignored [145896.240482] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [145896.247604] lnet: unknown parameter '#' ignored [145896.253316] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [145896.326176] Lustre: Lustre: Build Version: 2.15.4 [145896.384917] LNet: Using FastReg for registration [145896.600151] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [145898.149098] Key type lgssc registered [145898.303604] Lustre: Echo OBD driver; http://www.lustre.org/ [146489.520516] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [146510.482226] Lustre: DEBUG MARKER: server5: executing load_modules_local [146529.995788] device-mapper: ioctl: dmsetup[81672]: dm-3 (ost1_flakey) is created successfully [146531.219430] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [146531.308920] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [146531.321353] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [146532.393434] Lustre: lustre-OST0000: new disk, initializing [146532.400625] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [146532.423207] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [146532.442426] VFS: Open an exclusive opened block device for write dm-3. current [82056 tune2fs]. parent [82055 sh] [146533.274896] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [146537.017285] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [146546.943509] Lustre: DEBUG MARKER: Using TIMEOUT=20 [150814.056150] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [150815.645139] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [174821.615332] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [174822.758962] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [179541.636879] Lustre: ll_ost01_003: service thread pid 82594 was inactive for 40.011 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [179541.636881] Lustre: ll_ost01_006: service thread pid 83373 was inactive for 40.011 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [179541.636884] Pid: 81998, comm: ll_ost01_000 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [179541.636886] Lustre: ll_ost01_010: service thread pid 83377 was inactive for 40.011 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [179541.636891] Lustre: Skipped 1 previous similar message [179541.636892] Call Trace TBD: [179541.636894] Lustre: Skipped 4 previous similar messages [179541.658178] Lustre: Skipped 1 previous similar message [179541.658180] Pid: 82594, comm: ll_ost01_003 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [179541.658180] Call Trace TBD: [179541.747958] Pid: 83373, comm: ll_ost01_006 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [179541.760055] Call Trace TBD: [179541.908924] Lustre: lustre-OST0000: haven't heard from client 5caaec22-49f2-47cd-8fee-93a383cd7309 (at 192.168.0.81@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 0000000072568b5a, cur 1709811491 expire 1709811461 last 1709811443 [179541.934178] LustreError: 82594:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000001f5bf373 x1792832054025920/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [179541.959284] LustreError: 82594:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 2 previous similar messages [179570.157193] LustreError: 83377:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000000499e40f x1792832054027904/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [179570.182108] LustreError: 83377:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 13 previous similar messages [179576.004211] LustreError: 83376:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000020c6e559 x1792832054028160/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [179580.197424] LustreError: 83376:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000ecb3f41a x1792832054028544/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [179580.222571] LustreError: 83376:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 2 previous similar messages [179586.991941] LustreError: 83373:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000007bd7b22b x1792832054029248/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [179587.017216] LustreError: 83373:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 4 previous similar messages [179596.002768] LustreError: 83377:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000007c30370f x1792832054029696/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [179596.028039] LustreError: 83377:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 3 previous similar messages [179602.308545] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [179614.600960] LustreError: 81999:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000000432a514 x1792832054031168/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [179614.626188] LustreError: 81999:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 9 previous similar messages [179619.716150] Lustre: 80097:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709811562/real 1709811562] req@0000000015ed4c5f x1792832054030976/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709811569 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [179619.749890] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [179627.652087] Lustre: 83611:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709811571/real 1709811571] req@0000000080c3e188 x1792832054031744/t0(0) o39->lustre-MDT0001-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709811577 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [179634.308246] LustreError: 83611:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [179642.956792] Lustre: server umount lustre-OST0000 complete [179652.666456] device-mapper: ioctl: dmsetup[83977]: dm-3 (ost1_flakey) is removed successfully [179685.731230] Lustre: DEBUG MARKER: server5: executing unload_modules_local [179686.496183] Key type lgssc unregistered [179686.715667] LNet: 84550:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [179687.907562] LNet: Removed LNI 192.168.0.85@o2ib [179688.271677] Key type .llcrypt unregistered [179688.286332] Key type ._llcrypt unregistered [208287.563446] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [208289.276465] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [217790.649567] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [217791.302873] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [235394.221689] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [235394.231777] alg: No test for adler32 (adler32-zlib) [235394.979163] Key type ._llcrypt registered [235394.984380] Key type .llcrypt registered [235395.007190] Lustre: DEBUG MARKER: server5: executing set_hostid [235399.181981] Lustre: DEBUG MARKER: server5: executing load_modules_local [235399.558278] lnet: unknown parameter '#' ignored [235399.564109] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [235399.571229] lnet: unknown parameter '#' ignored [235399.576929] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [235399.653401] Lustre: Lustre: Build Version: 2.15.4 [235399.714419] LNet: Using FastReg for registration [235399.930546] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [235401.479001] Key type lgssc registered [235401.642676] Lustre: Echo OBD driver; http://www.lustre.org/ [237678.754557] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237709.314120] LDISKFS-fs (nvme1n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237735.378016] LDISKFS-fs (nvme2n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237761.470115] LDISKFS-fs (nvme3n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237782.399663] Lustre: DEBUG MARKER: server5: executing load_modules_local [237846.356913] device-mapper: ioctl: dmsetup[91408]: dm-3 (ost1_flakey) is created successfully [237847.569179] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237847.661821] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [237847.674340] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [237848.750294] Lustre: lustre-OST0000: new disk, initializing [237848.757496] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [237848.786462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [237848.806989] VFS: Open an exclusive opened block device for write dm-3. current [91797 tune2fs]. parent [91796 sh] [237849.665966] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [237850.798406] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000400000400-0x0000000440000400]:0:ost] [237860.072079] device-mapper: ioctl: dmsetup[92772]: dm-4 (ost3_flakey) is created successfully [237861.299593] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237861.377258] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [237861.389883] Lustre: lustre-OST0002-osd: enabled 'large_dir' feature on device /dev/mapper/ost3_flakey [237861.414726] Lustre: lustre-OST0002: new disk, initializing [237861.421976] Lustre: srv-lustre-OST0002: No data found on store. Initialize space: rc = -61 [237861.451101] Lustre: lustre-OST0002: Imperative Recovery not enabled, recovery window 60-180 [237861.470594] VFS: Open an exclusive opened block device for write dm-4. current [93082 tune2fs]. parent [93081 sh] [237862.283371] Lustre: cli-lustre-OST0002-super: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:2:ost] [237862.316004] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [237871.682844] device-mapper: ioctl: dmsetup[94050]: dm-5 (ost5_flakey) is created successfully [237872.913554] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237873.002506] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [237873.015406] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [237873.040201] Lustre: lustre-OST0004: new disk, initializing [237873.047190] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [237873.070118] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [237873.090118] VFS: Open an exclusive opened block device for write dm-5. current [94361 tune2fs]. parent [94360 sh] [237873.923806] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [237874.937606] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000500000400-0x0000000540000400]:4:ost] [237883.298354] device-mapper: ioctl: dmsetup[95338]: dm-6 (ost7_flakey) is created successfully [237884.541856] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [237884.628675] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [237884.641844] Lustre: lustre-OST0006-osd: enabled 'large_dir' feature on device /dev/mapper/ost7_flakey [237884.667158] Lustre: lustre-OST0006: new disk, initializing [237884.674374] Lustre: srv-lustre-OST0006: No data found on store. Initialize space: rc = -61 [237884.698023] Lustre: lustre-OST0006: Imperative Recovery not enabled, recovery window 60-180 [237884.718774] VFS: Open an exclusive opened block device for write dm-6. current [95647 tune2fs]. parent [95646 sh] [237885.546931] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [237892.490033] Lustre: cli-lustre-OST0006-super: Allocated super-sequence [0x0000000580000400-0x00000005c0000400]:6:ost] [237898.275173] Lustre: DEBUG MARKER: Using TIMEOUT=20 [244745.530353] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [244745.549278] Lustre: Skipped 2 previous similar messages [244750.654354] Lustre: lustre-MDT0001-lwp-OST0002: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [244750.673152] Lustre: Skipped 4 previous similar messages [244757.818118] Lustre: 88488:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709876700/real 1709876700] req@00000000bf8abfa2 x1792925921232576/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709876707 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:3.0' [244757.851596] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [244762.440634] Lustre: lustre-OST0004: haven't heard from client 5fce0288-2f8a-401b-8111-fd32915272b4 (at 192.168.0.81@o2ib) in 49 seconds. I think it's dead, and I am evicting it. exp 0000000018ef3fc5, cur 1709876712 expire 1709876682 last 1709876663 [244762.938228] Lustre: lustre-MDT0002-lwp-OST0002: Connection to lustre-MDT0002 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [244762.952608] Lustre: lustre-OST0006: haven't heard from client 5fce0288-2f8a-401b-8111-fd32915272b4 (at 192.168.0.81@o2ib) in 50 seconds. I think it's dead, and I am evicting it. exp 0000000057be4615, cur 1709876713 expire 1709876683 last 1709876663 [244762.957010] Lustre: Skipped 3 previous similar messages [244762.981601] Lustre: Skipped 1 previous similar message [244773.178106] Lustre: lustre-MDT0003-lwp-OST0000: Connection to lustre-MDT0003 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [244773.196834] Lustre: Skipped 3 previous similar messages [244779.322140] Lustre: lustre-MDT0004-lwp-OST0000: Connection to lustre-MDT0004 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [244779.340866] Lustre: Skipped 3 previous similar messages [244789.562006] Lustre: lustre-MDT0005-lwp-OST0000: Connection to lustre-MDT0005 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [244789.580733] Lustre: Skipped 3 previous similar messages [244815.417605] Lustre: 96839:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709876759/real 1709876759] req@0000000023cc867a x1792925921263616/t0(0) o39->lustre-MDT0007-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709876765 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [244815.455844] LustreError: 96839:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [244815.473232] Lustre: server umount lustre-OST0000 complete [244817.025306] device-mapper: ioctl: dmsetup[97201]: dm-3 (ost1_flakey) is removed successfully [244818.105577] Lustre: 88500:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709876761/real 1709876761] req@000000007c2d82ce x1792925921264384/t0(0) o400->lustre-MDT0007-lwp-OST0006@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709876768 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [244818.139535] Lustre: lustre-MDT0007-lwp-OST0006: Connection to lustre-MDT0007 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [244818.158260] Lustre: Skipped 4 previous similar messages [244821.050387] LustreError: 97489:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [244821.075710] Lustre: server umount lustre-OST0002 complete [244822.651833] device-mapper: ioctl: dmsetup[97850]: dm-4 (ost3_flakey) is removed successfully [244823.353520] Lustre: 88478:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709876766/real 1709876766] req@000000002f0fb94b x1792925921264576/t0(0) o400->lustre-MDT0007-lwp-OST0004@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709876773 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [244823.353522] Lustre: 88479:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709876766/real 1709876766] req@00000000d26bdf7a x1792925921264640/t0(0) o400->lustre-MDT0007-lwp-OST0006@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709876773 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [244823.353526] Lustre: 88479:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 1 previous similar message [244823.387575] Lustre: 88478:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 1 previous similar message [244826.708187] LustreError: 98136:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [244826.727927] Lustre: server umount lustre-OST0004 complete [244828.300209] device-mapper: ioctl: dmsetup[98500]: dm-5 (ost5_flakey) is removed successfully [244832.335963] LustreError: 98786:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [244832.396447] Lustre: server umount lustre-OST0006 complete [244833.920188] device-mapper: ioctl: dmsetup[99150]: dm-6 (ost7_flakey) is removed successfully [244840.602931] Lustre: DEBUG MARKER: server5: executing unload_modules_local [244841.430164] Key type lgssc unregistered [244841.637455] LNet: 99720:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [244842.809366] LNet: Removed LNI 192.168.0.85@o2ib [244843.201396] Key type .llcrypt unregistered [244843.215792] Key type ._llcrypt unregistered [244861.681623] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [244861.692431] alg: No test for adler32 (adler32-zlib) [244862.441287] Key type ._llcrypt registered [244862.447146] Key type .llcrypt registered [244862.470232] Lustre: DEBUG MARKER: server5: executing set_hostid [244866.631669] Lustre: DEBUG MARKER: server5: executing load_modules_local [244867.026403] lnet: unknown parameter '#' ignored [244867.032336] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [244867.039525] lnet: unknown parameter '#' ignored [244867.045307] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [244867.120268] Lustre: Lustre: Build Version: 2.15.4 [244867.181685] LNet: Using FastReg for registration [244867.391996] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [244868.937107] Key type lgssc registered [244869.096756] Lustre: Echo OBD driver; http://www.lustre.org/ [247175.313617] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247201.480823] LDISKFS-fs (nvme1n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247228.532924] LDISKFS-fs (nvme2n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247255.281347] LDISKFS-fs (nvme3n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247276.228586] Lustre: DEBUG MARKER: server5: executing load_modules_local [247340.288378] device-mapper: ioctl: dmsetup[104850]: dm-3 (ost1_flakey) is created successfully [247341.507869] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247341.604135] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [247341.617100] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [247342.693163] Lustre: lustre-OST0000: new disk, initializing [247342.700457] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [247342.727750] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [247342.745656] VFS: Open an exclusive opened block device for write dm-3. current [105240 tune2fs]. parent [105239 sh] [247343.508332] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000400000400-0x0000000440000400]:0:ost] [247343.583936] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [247353.965018] device-mapper: ioctl: dmsetup[106210]: dm-4 (ost3_flakey) is created successfully [247355.191690] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247355.280156] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [247355.292893] Lustre: lustre-OST0002-osd: enabled 'large_dir' feature on device /dev/mapper/ost3_flakey [247355.317097] Lustre: lustre-OST0002: new disk, initializing [247355.324139] Lustre: srv-lustre-OST0002: No data found on store. Initialize space: rc = -61 [247355.344660] Lustre: lustre-OST0002: Imperative Recovery not enabled, recovery window 60-180 [247355.361348] VFS: Open an exclusive opened block device for write dm-4. current [106522 tune2fs]. parent [106521 sh] [247356.189714] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [247357.352091] Lustre: cli-lustre-OST0002-super: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:2:ost] [247365.578871] device-mapper: ioctl: dmsetup[107494]: dm-5 (ost5_flakey) is created successfully [247366.820394] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247366.908244] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [247366.921184] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [247366.946400] Lustre: lustre-OST0004: new disk, initializing [247366.953447] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [247366.976481] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [247366.993394] VFS: Open an exclusive opened block device for write dm-5. current [107806 tune2fs]. parent [107805 sh] [247367.821238] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [247373.875980] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000500000400-0x0000000540000400]:4:ost] [247377.190708] device-mapper: ioctl: dmsetup[108778]: dm-6 (ost7_flakey) is created successfully [247378.426079] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [247378.515295] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [247378.528556] Lustre: lustre-OST0006-osd: enabled 'large_dir' feature on device /dev/mapper/ost7_flakey [247378.553982] Lustre: lustre-OST0006: new disk, initializing [247378.561264] Lustre: srv-lustre-OST0006: No data found on store. Initialize space: rc = -61 [247378.584791] Lustre: lustre-OST0006: Imperative Recovery not enabled, recovery window 60-180 [247378.605487] VFS: Open an exclusive opened block device for write dm-6. current [109090 tune2fs]. parent [109089 sh] [247379.443512] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [247383.603692] Lustre: cli-lustre-OST0006-super: Allocated super-sequence [0x0000000580000400-0x00000005c0000400]:6:ost] [247392.164896] Lustre: DEBUG MARKER: Using TIMEOUT=20 [255539.673822] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [255539.692846] Lustre: Skipped 3 previous similar messages [255548.907285] Lustre: lustre-OST0006: haven't heard from client fd95df74-9ed1-47c6-a788-48295fcc49b6 (at 192.168.0.81@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 000000005c5636eb, cur 1709887499 expire 1709887469 last 1709887452 [255550.300217] Lustre: lustre-OST0004: haven't heard from client fd95df74-9ed1-47c6-a788-48295fcc49b6 (at 192.168.0.81@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 00000000c95462ce, cur 1709887500 expire 1709887470 last 1709887452 [255550.324819] Lustre: Skipped 1 previous similar message [255555.033651] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [255555.053113] Lustre: Skipped 3 previous similar messages [255560.153612] Lustre: lustre-MDT0002-lwp-OST0000: Connection to lustre-MDT0002 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [255560.172462] Lustre: Skipped 3 previous similar messages [255562.201404] Lustre: 101969:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709887505/real 1709887505] req@000000003fb399dd x1792935887229440/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709887512 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [255562.234878] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [255573.465492] Lustre: lustre-MDT0003-lwp-OST0000: Connection to lustre-MDT0003 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [255573.484298] Lustre: Skipped 3 previous similar messages [255578.585510] Lustre: lustre-MDT0004-lwp-OST0000: Connection to lustre-MDT0004 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [255578.604273] Lustre: Skipped 3 previous similar messages [255589.849328] Lustre: lustre-MDT0005-lwp-OST0000: Connection to lustre-MDT0005 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [255589.868335] Lustre: Skipped 3 previous similar messages [255615.960943] Lustre: 110414:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709887560/real 1709887560] req@0000000016eae47a x1792935887260480/t0(0) o39->lustre-MDT0007-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709887566 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [255615.999066] LustreError: 110414:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [255616.016913] Lustre: server umount lustre-OST0000 complete [255617.604871] device-mapper: ioctl: dmsetup[110774]: dm-3 (ost1_flakey) is removed successfully [255618.520907] Lustre: 101915:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709887561/real 1709887561] req@00000000ae6091c6 x1792935887260736/t0(0) o400->lustre-MDT0007-lwp-OST0002@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709887568 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:1.0' [255618.520924] Lustre: lustre-MDT0007-lwp-OST0004: Connection to lustre-MDT0007 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [255618.555257] Lustre: 101915:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [255618.574080] Lustre: Skipped 5 previous similar messages [255621.593926] LustreError: 111061:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [255621.605147] LustreError: 111061:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [255621.627183] Lustre: server umount lustre-OST0002 complete [255623.209381] device-mapper: ioctl: dmsetup[111427]: dm-4 (ost3_flakey) is removed successfully [255623.640861] Lustre: 101919:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709887566/real 1709887566] req@00000000b7f687b1 x1792935887261376/t0(0) o400->lustre-MDT0007-lwp-OST0004@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709887573 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:1.0' [255623.675167] Lustre: 101919:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 1 previous similar message [255627.212903] LustreError: 111710:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [255627.224157] LustreError: 111710:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [255627.243743] Lustre: server umount lustre-OST0004 complete [255628.829774] device-mapper: ioctl: dmsetup[112075]: dm-5 (ost5_flakey) is removed successfully [255632.897013] LustreError: 112359:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [255632.908422] LustreError: 112359:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [255632.976059] Lustre: server umount lustre-OST0006 complete [255634.523975] device-mapper: ioctl: dmsetup[112724]: dm-6 (ost7_flakey) is removed successfully [255641.039646] Lustre: DEBUG MARKER: server5: executing unload_modules_local [255641.857558] Key type lgssc unregistered [255642.056803] LNet: 113293:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [255643.224697] LNet: Removed LNI 192.168.0.85@o2ib [255643.568873] Key type .llcrypt unregistered [255643.583276] Key type ._llcrypt unregistered [255662.169909] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [255662.180458] alg: No test for adler32 (adler32-zlib) [255662.928587] Key type ._llcrypt registered [255662.933941] Key type .llcrypt registered [255662.957078] Lustre: DEBUG MARKER: server5: executing set_hostid [255667.025376] Lustre: DEBUG MARKER: server5: executing load_modules_local [255667.416869] lnet: unknown parameter '#' ignored [255667.423018] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [255667.430212] lnet: unknown parameter '#' ignored [255667.435995] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [255667.512995] Lustre: Lustre: Build Version: 2.15.4 [255667.576083] LNet: Using FastReg for registration [255667.781682] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [255669.324449] Key type lgssc registered [255669.478513] Lustre: Echo OBD driver; http://www.lustre.org/ [257976.798423] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258002.958677] LDISKFS-fs (nvme1n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258029.570463] LDISKFS-fs (nvme2n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258057.496725] LDISKFS-fs (nvme3n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258078.996337] Lustre: DEBUG MARKER: server5: executing load_modules_local [258142.804722] device-mapper: ioctl: dmsetup[118419]: dm-3 (ost1_flakey) is created successfully [258144.021558] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258144.111704] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [258144.124229] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [258145.196118] Lustre: lustre-OST0000: new disk, initializing [258145.203217] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [258145.230327] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [258145.249470] VFS: Open an exclusive opened block device for write dm-3. current [118811 tune2fs]. parent [118810 sh] [258146.077364] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [258147.032535] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000400000400-0x0000000440000400]:0:ost] [258156.433764] device-mapper: ioctl: dmsetup[119787]: dm-4 (ost3_flakey) is created successfully [258157.667047] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258157.756486] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [258157.769451] Lustre: lustre-OST0002-osd: enabled 'large_dir' feature on device /dev/mapper/ost3_flakey [258157.794043] Lustre: lustre-OST0002: new disk, initializing [258157.801085] Lustre: srv-lustre-OST0002: No data found on store. Initialize space: rc = -61 [258157.824810] Lustre: lustre-OST0002: Imperative Recovery not enabled, recovery window 60-180 [258157.844176] VFS: Open an exclusive opened block device for write dm-4. current [120097 tune2fs]. parent [120096 sh] [258158.682438] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [258159.943783] Lustre: cli-lustre-OST0002-super: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:2:ost] [258168.013144] device-mapper: ioctl: dmsetup[121067]: dm-5 (ost5_flakey) is created successfully [258169.241520] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258169.320494] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [258169.333545] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [258169.358735] Lustre: lustre-OST0004: new disk, initializing [258169.365826] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [258169.388468] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [258169.405553] VFS: Open an exclusive opened block device for write dm-5. current [121377 tune2fs]. parent [121376 sh] [258170.226597] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [258173.910803] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000500000400-0x0000000540000400]:4:ost] [258179.585990] device-mapper: ioctl: dmsetup[122353]: dm-6 (ost7_flakey) is created successfully [258180.823970] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [258180.903315] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [258180.916063] Lustre: lustre-OST0006-osd: enabled 'large_dir' feature on device /dev/mapper/ost7_flakey [258180.940809] Lustre: lustre-OST0006: new disk, initializing [258180.948110] Lustre: srv-lustre-OST0006: No data found on store. Initialize space: rc = -61 [258180.972003] Lustre: lustre-OST0006: Imperative Recovery not enabled, recovery window 60-180 [258180.992056] VFS: Open an exclusive opened block device for write dm-6. current [122662 tune2fs]. parent [122661 sh] [258181.815472] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [258186.586834] Lustre: cli-lustre-OST0006-super: Allocated super-sequence [0x0000000580000400-0x00000005c0000400]:6:ost] [258194.534338] Lustre: DEBUG MARKER: Using TIMEOUT=20 [267112.836985] Lustre: lustre-OST0002: haven't heard from client e1897424-fd25-459c-a1ec-0725baadea51 (at 192.168.0.82@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 00000000585257cc, cur 1709899063 expire 1709899033 last 1709899016 [267115.378446] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [267115.397721] Lustre: Skipped 2 previous similar messages [267125.618402] Lustre: lustre-MDT0001-lwp-OST0002: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [267125.637346] Lustre: Skipped 4 previous similar messages [267127.666208] Lustre: 115494:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709899070/real 1709899070] req@00000000baf24f1e x1792947280617216/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709899077 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [267127.699868] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [267132.786320] Lustre: lustre-MDT0002-lwp-OST0002: Connection to lustre-MDT0002 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [267132.805194] Lustre: Skipped 3 previous similar messages [267144.050244] Lustre: lustre-MDT0003-lwp-OST0000: Connection to lustre-MDT0003 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [267144.069093] Lustre: Skipped 3 previous similar messages [267155.314140] Lustre: lustre-MDT0004-lwp-OST0000: Connection to lustre-MDT0004 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [267155.333243] Lustre: Skipped 3 previous similar messages [267171.698006] Lustre: lustre-MDT0006-lwp-OST0000: Connection to lustre-MDT0006 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [267171.716985] Lustre: Skipped 7 previous similar messages [267189.105650] Lustre: 115481:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709899132/real 1709899132] req@0000000097dbcff3 x1792947280650304/t0(0) o400->lustre-MDT0007-lwp-OST0002@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709899139 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [267189.105660] Lustre: lustre-MDT0007-lwp-OST0004: Connection to lustre-MDT0007 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [267189.115098] Lustre: server umount lustre-OST0000 complete [267189.139729] Lustre: 115481:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [267189.158523] Lustre: Skipped 1 previous similar message [267190.129650] Lustre: 115520:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709899133/real 1709899133] req@000000007d765e28 x1792947280651072/t0(0) o400->lustre-MDT0007-lwp-OST0002@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709899140 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [267190.163952] Lustre: 115520:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [267190.776113] device-mapper: ioctl: dmsetup[124277]: dm-3 (ost1_flakey) is removed successfully [267194.225613] Lustre: 115484:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709899137/real 1709899137] req@0000000075260ad2 x1792947280651776/t0(0) o400->lustre-MDT0007-lwp-OST0004@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709899144 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [267194.259904] Lustre: 115484:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 1 previous similar message [267194.839785] Lustre: server umount lustre-OST0002 complete [267196.496493] device-mapper: ioctl: dmsetup[124926]: dm-4 (ost3_flakey) is removed successfully [267200.496824] Lustre: server umount lustre-OST0004 complete [267202.121604] device-mapper: ioctl: dmsetup[125573]: dm-5 (ost5_flakey) is removed successfully [267206.185167] Lustre: server umount lustre-OST0006 complete [267207.785628] device-mapper: ioctl: dmsetup[126221]: dm-6 (ost7_flakey) is removed successfully [267214.172680] Lustre: DEBUG MARKER: server5: executing unload_modules_local [267214.874617] Key type lgssc unregistered [267215.057516] LNet: 126790:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [267216.241420] LNet: Removed LNI 192.168.0.85@o2ib [267216.601670] Key type .llcrypt unregistered [267216.616469] Key type ._llcrypt unregistered [267235.101385] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [267235.111839] alg: No test for adler32 (adler32-zlib) [267235.861393] Key type ._llcrypt registered [267235.867252] Key type .llcrypt registered [267235.890112] Lustre: DEBUG MARKER: server5: executing set_hostid [267240.068751] Lustre: DEBUG MARKER: server5: executing load_modules_local [267240.463956] lnet: unknown parameter '#' ignored [267240.470147] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [267240.477351] lnet: unknown parameter '#' ignored [267240.483125] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [267240.558186] Lustre: Lustre: Build Version: 2.15.4 [267240.616880] LNet: Using FastReg for registration [267240.834054] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [267242.377170] Key type lgssc registered [267242.534569] Lustre: Echo OBD driver; http://www.lustre.org/ [269549.831840] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269576.867249] LDISKFS-fs (nvme1n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269603.912584] LDISKFS-fs (nvme2n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269630.989542] LDISKFS-fs (nvme3n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269652.414826] Lustre: DEBUG MARKER: server5: executing load_modules_local [269716.324115] device-mapper: ioctl: dmsetup[131933]: dm-3 (ost1_flakey) is created successfully [269717.554234] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269717.653069] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [269717.665794] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [269718.736794] Lustre: lustre-OST0000: new disk, initializing [269718.743912] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [269718.770360] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [269718.790042] VFS: Open an exclusive opened block device for write dm-3. current [132323 tune2fs]. parent [132322 sh] [269719.120787] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000400000400-0x0000000440000400]:0:ost] [269719.618861] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [269729.977414] device-mapper: ioctl: dmsetup[133299]: dm-4 (ost3_flakey) is created successfully [269731.220854] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269731.309067] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [269731.322011] Lustre: lustre-OST0002-osd: enabled 'large_dir' feature on device /dev/mapper/ost3_flakey [269731.346059] Lustre: lustre-OST0002: new disk, initializing [269731.353186] Lustre: srv-lustre-OST0002: No data found on store. Initialize space: rc = -61 [269731.376362] Lustre: lustre-OST0002: Imperative Recovery not enabled, recovery window 60-180 [269731.396527] VFS: Open an exclusive opened block device for write dm-4. current [133611 tune2fs]. parent [133610 sh] [269731.809215] Lustre: cli-lustre-OST0002-super: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:2:ost] [269732.237732] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [269741.557988] device-mapper: ioctl: dmsetup[134585]: dm-5 (ost5_flakey) is created successfully [269742.782520] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269742.860064] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [269742.872803] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [269742.897819] Lustre: lustre-OST0004: new disk, initializing [269742.905174] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [269742.929203] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [269742.946125] VFS: Open an exclusive opened block device for write dm-5. current [134897 tune2fs]. parent [134896 sh] [269743.772855] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [269747.425047] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000500000400-0x0000000540000400]:4:ost] [269753.123041] device-mapper: ioctl: dmsetup[135870]: dm-6 (ost7_flakey) is created successfully [269754.365229] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [269754.461068] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [269754.474035] Lustre: lustre-OST0006-osd: enabled 'large_dir' feature on device /dev/mapper/ost7_flakey [269754.499767] Lustre: lustre-OST0006: new disk, initializing [269754.506798] Lustre: srv-lustre-OST0006: No data found on store. Initialize space: rc = -61 [269754.528489] Lustre: lustre-OST0006: Imperative Recovery not enabled, recovery window 60-180 [269754.545506] VFS: Open an exclusive opened block device for write dm-6. current [136182 tune2fs]. parent [136181 sh] [269755.377205] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [269756.399804] Lustre: cli-lustre-OST0006-super: Allocated super-sequence [0x0000000580000400-0x00000005c0000400]:6:ost] [269768.102161] Lustre: DEBUG MARKER: Using TIMEOUT=20 [272774.587930] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [272774.904072] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [278979.611814] Lustre: lustre-OST0000: haven't heard from client ffec1193-bdd4-4213-bcc8-906be32814ec (at 192.168.0.82@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 000000003c3674e5, cur 1709910930 expire 1709910900 last 1709910883 [278981.407992] Lustre: lustre-OST0002: haven't heard from client ffec1193-bdd4-4213-bcc8-906be32814ec (at 192.168.0.82@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 000000006508253e, cur 1709910931 expire 1709910901 last 1709910883 [278981.432733] Lustre: Skipped 5 previous similar messages [279001.097012] Lustre: lustre-MDT0000-lwp-OST0002: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [279001.116064] Lustre: Skipped 3 previous similar messages [279011.336947] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [279011.355961] Lustre: Skipped 3 previous similar messages [279013.384772] Lustre: 129017:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709910956/real 1709910956] req@00000000036e91ae x1792959505225600/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709910963 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [279013.418632] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [279019.528914] Lustre: lustre-MDT0002-lwp-OST0000: Connection to lustre-MDT0002 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [279019.547906] Lustre: Skipped 3 previous similar messages [279029.768933] Lustre: lustre-MDT0003-lwp-OST0000: Connection to lustre-MDT0003 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [279029.788087] Lustre: Skipped 3 previous similar messages [279040.008822] Lustre: lustre-MDT0004-lwp-OST0002: Connection to lustre-MDT0004 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [279040.027836] Lustre: Skipped 3 previous similar messages [279051.272621] Lustre: lustre-MDT0005-lwp-OST0000: Connection to lustre-MDT0005 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [279051.291410] Lustre: Skipped 3 previous similar messages [279076.104210] Lustre: 137462:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709911020/real 1709911020] req@00000000645bc25d x1792959505258752/t0(0) o39->lustre-MDT0007-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709911026 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [279076.142713] LustreError: 137462:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [279076.160655] Lustre: server umount lustre-OST0000 complete [279077.843667] device-mapper: ioctl: dmsetup[137823]: dm-3 (ost1_flakey) is removed successfully [279078.920188] Lustre: 129001:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709911022/real 1709911022] req@00000000d4d2eb19 x1792959505259520/t0(0) o400->lustre-MDT0007-lwp-OST0006@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709911029 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [279078.920190] Lustre: 129023:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709911022/real 1709911022] req@000000006c690104 x1792959505259008/t0(0) o400->lustre-MDT0007-lwp-OST0002@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709911029 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [279078.920195] Lustre: 129023:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 1 previous similar message [279078.920199] Lustre: lustre-MDT0007-lwp-OST0004: Connection to lustre-MDT0007 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [279078.920200] Lustre: Skipped 4 previous similar messages [279078.954524] Lustre: 129001:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 1 previous similar message [279082.062916] LustreError: 138109:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [279082.074269] LustreError: 138109:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [279082.094810] Lustre: server umount lustre-OST0002 complete [279083.785708] device-mapper: ioctl: dmsetup[138475]: dm-4 (ost3_flakey) is removed successfully [279084.040138] Lustre: 129003:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709911027/real 1709911027] req@000000006c401572 x1792959505259648/t0(0) o400->lustre-MDT0007-lwp-OST0004@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709911034 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [279084.074782] Lustre: 129003:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 1 previous similar message [279088.028824] LustreError: 138760:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [279088.040230] LustreError: 138760:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [279088.066543] Lustre: server umount lustre-OST0004 complete [279089.798435] device-mapper: ioctl: dmsetup[139121]: dm-5 (ost5_flakey) is removed successfully [279093.991762] LustreError: 139405:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [279094.003143] LustreError: 139405:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [279094.081538] Lustre: server umount lustre-OST0006 complete [279095.725772] device-mapper: ioctl: dmsetup[139771]: dm-6 (ost7_flakey) is removed successfully [279102.336260] Lustre: DEBUG MARKER: server5: executing unload_modules_local [279103.101097] Key type lgssc unregistered [279103.272058] LNet: 140342:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [279104.455972] LNet: Removed LNI 192.168.0.85@o2ib [279104.800109] Key type .llcrypt unregistered [279104.814691] Key type ._llcrypt unregistered [279123.512123] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [279123.522826] alg: No test for adler32 (adler32-zlib) [279124.271894] Key type ._llcrypt registered [279124.277731] Key type .llcrypt registered [279124.300848] Lustre: DEBUG MARKER: server5: executing set_hostid [279128.447966] Lustre: DEBUG MARKER: server5: executing load_modules_local [279128.858425] lnet: unknown parameter '#' ignored [279128.864569] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [279128.871766] lnet: unknown parameter '#' ignored [279128.877541] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [279128.952308] Lustre: Lustre: Build Version: 2.15.4 [279129.010989] LNet: Using FastReg for registration [279129.223187] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [279130.767721] Key type lgssc registered [279130.926698] Lustre: Echo OBD driver; http://www.lustre.org/ [281435.873934] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281463.523400] LDISKFS-fs (nvme1n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281493.573657] LDISKFS-fs (nvme2n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281521.260077] LDISKFS-fs (nvme3n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281543.082798] Lustre: DEBUG MARKER: server5: executing load_modules_local [281606.836822] device-mapper: ioctl: dmsetup[145447]: dm-3 (ost1_flakey) is created successfully [281608.051155] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281608.153569] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [281608.166366] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [281609.243681] Lustre: lustre-OST0000: new disk, initializing [281609.250857] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [281609.278423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [281609.295333] VFS: Open an exclusive opened block device for write dm-3. current [145836 tune2fs]. parent [145835 sh] [281610.126176] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [281614.987415] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000400000400-0x0000000440000400]:0:ost] [281620.475603] device-mapper: ioctl: dmsetup[146812]: dm-4 (ost3_flakey) is created successfully [281621.737122] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281621.823092] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [281621.836016] Lustre: lustre-OST0002-osd: enabled 'large_dir' feature on device /dev/mapper/ost3_flakey [281621.860875] Lustre: lustre-OST0002: new disk, initializing [281621.868226] Lustre: srv-lustre-OST0002: No data found on store. Initialize space: rc = -61 [281621.891500] Lustre: lustre-OST0002: Imperative Recovery not enabled, recovery window 60-180 [281621.911932] VFS: Open an exclusive opened block device for write dm-4. current [147124 tune2fs]. parent [147123 sh] [281621.998935] Lustre: cli-lustre-OST0002-super: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:2:ost] [281622.741225] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [281632.090085] device-mapper: ioctl: dmsetup[148097]: dm-5 (ost5_flakey) is created successfully [281633.314536] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281633.403901] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [281633.416895] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [281633.441543] Lustre: lustre-OST0004: new disk, initializing [281633.448572] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [281633.471279] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [281633.488131] VFS: Open an exclusive opened block device for write dm-5. current [148406 tune2fs]. parent [148405 sh] [281634.313849] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [281635.846814] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000500000400-0x0000000540000400]:4:ost] [281643.660541] device-mapper: ioctl: dmsetup[149385]: dm-6 (ost7_flakey) is created successfully [281644.883506] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [281644.971461] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [281644.984432] Lustre: lustre-OST0006-osd: enabled 'large_dir' feature on device /dev/mapper/ost7_flakey [281645.009740] Lustre: lustre-OST0006: new disk, initializing [281645.016771] Lustre: srv-lustre-OST0006: No data found on store. Initialize space: rc = -61 [281645.036467] Lustre: lustre-OST0006: Imperative Recovery not enabled, recovery window 60-180 [281645.053534] VFS: Open an exclusive opened block device for write dm-6. current [149695 tune2fs]. parent [149694 sh] [281645.884538] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [281646.423654] Lustre: cli-lustre-OST0006-super: Allocated super-sequence [0x0000000580000400-0x00000005c0000400]:6:ost] [281658.564622] Lustre: DEBUG MARKER: Using TIMEOUT=20 [303184.565767] Lustre: lustre-OST0004: haven't heard from client 744c6f95-a11b-4117-b31a-9101930ea37f (at 192.168.0.81@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 00000000372af3bf, cur 1709935135 expire 1709935105 last 1709935088 [303185.478174] Lustre: lustre-OST0000: haven't heard from client 744c6f95-a11b-4117-b31a-9101930ea37f (at 192.168.0.81@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 000000002f577239, cur 1709935136 expire 1709935106 last 1709935088 [303185.502957] Lustre: Skipped 1 previous similar message [303186.502168] Lustre: lustre-OST0006: haven't heard from client 744c6f95-a11b-4117-b31a-9101930ea37f (at 192.168.0.81@o2ib) in 49 seconds. I think it's dead, and I am evicting it. exp 00000000d7993358, cur 1709935137 expire 1709935107 last 1709935088 [303186.527165] Lustre: Skipped 3 previous similar messages [303225.651648] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [303225.670956] Lustre: Skipped 2 previous similar messages [303230.771603] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [303230.790441] Lustre: Skipped 4 previous similar messages [303237.939310] Lustre: 142582:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709935181/real 1709935181] req@00000000166a7df1 x1792972225553984/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709935188 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:4.0' [303237.972815] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [303243.059412] Lustre: lustre-MDT0002-lwp-OST0000: Connection to lustre-MDT0002 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [303243.078206] Lustre: Skipped 3 previous similar messages [303249.203409] Lustre: lustre-MDT0003-lwp-OST0000: Connection to lustre-MDT0003 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [303249.222177] Lustre: Skipped 1 previous similar message [303260.467390] Lustre: lustre-MDT0004-lwp-OST0002: Connection to lustre-MDT0004 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [303260.486176] Lustre: Skipped 1 previous similar message [303270.707240] Lustre: lustre-MDT0005-lwp-OST0000: Connection to lustre-MDT0005 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [303270.725998] Lustre: Skipped 3 previous similar messages [303297.586781] Lustre: 151366:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709935242/real 1709935242] req@00000000aae9f185 x1792972225586048/t0(0) o39->lustre-MDT0007-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709935248 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [303297.632208] Lustre: server umount lustre-OST0000 complete [303298.354791] Lustre: lustre-MDT0007-lwp-OST0006: Connection to lustre-MDT0007 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [303298.373683] Lustre: Skipped 6 previous similar messages [303299.250768] Lustre: 142537:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709935242/real 1709935242] req@0000000034d8faa3 x1792972225586304/t0(0) o400->lustre-MDT0007-lwp-OST0006@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709935249 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [303299.284850] Lustre: 142537:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [303299.394018] device-mapper: ioctl: dmsetup[151729]: dm-3 (ost1_flakey) is removed successfully [303302.450745] Lustre: 142538:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709935246/real 1709935246] req@00000000a77e52b0 x1792972225586368/t0(0) o400->lustre-MDT0007-lwp-OST0002@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709935253 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [303302.484939] Lustre: 142538:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [303303.606687] Lustre: server umount lustre-OST0002 complete [303305.357832] device-mapper: ioctl: dmsetup[152375]: dm-4 (ost3_flakey) is removed successfully [303309.571470] Lustre: server umount lustre-OST0004 complete [303311.327087] device-mapper: ioctl: dmsetup[153035]: dm-5 (ost5_flakey) is removed successfully [303315.584776] Lustre: server umount lustre-OST0006 complete [303317.260797] device-mapper: ioctl: dmsetup[153681]: dm-6 (ost7_flakey) is removed successfully [303323.886001] Lustre: DEBUG MARKER: server5: executing unload_modules_local [303324.571080] Key type lgssc unregistered [303324.758610] LNet: 154252:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [303325.938535] LNet: Removed LNI 192.168.0.85@o2ib [303326.318789] Key type .llcrypt unregistered [303326.333276] Key type ._llcrypt unregistered [303344.993032] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [303345.003518] alg: No test for adler32 (adler32-zlib) [303345.750473] Key type ._llcrypt registered [303345.756025] Key type .llcrypt registered [303345.779256] Lustre: DEBUG MARKER: server5: executing set_hostid [303349.891905] Lustre: DEBUG MARKER: server5: executing load_modules_local [303350.279238] lnet: unknown parameter '#' ignored [303350.285134] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [303350.292336] lnet: unknown parameter '#' ignored [303350.298115] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [303350.372143] Lustre: Lustre: Build Version: 2.15.4 [303350.431329] LNet: Using FastReg for registration [303350.637619] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [303352.182293] Key type lgssc registered [303352.338764] Lustre: Echo OBD driver; http://www.lustre.org/ [305660.557028] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305689.714261] LDISKFS-fs (nvme1n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305718.864169] LDISKFS-fs (nvme2n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305747.993990] LDISKFS-fs (nvme3n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305770.467063] Lustre: DEBUG MARKER: server5: executing load_modules_local [305834.466023] device-mapper: ioctl: dmsetup[159383]: dm-3 (ost1_flakey) is created successfully [305835.681044] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305835.781356] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [305835.794042] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [305836.865969] Lustre: lustre-OST0000: new disk, initializing [305836.873025] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [305836.900238] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [305836.917251] VFS: Open an exclusive opened block device for write dm-3. current [159774 tune2fs]. parent [159773 sh] [305837.746195] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [305838.258151] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000400000400-0x0000000440000400]:0:ost] [305848.112459] device-mapper: ioctl: dmsetup[160748]: dm-4 (ost3_flakey) is created successfully [305849.340144] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305849.429552] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [305849.442474] Lustre: lustre-OST0002-osd: enabled 'large_dir' feature on device /dev/mapper/ost3_flakey [305849.467393] Lustre: lustre-OST0002: new disk, initializing [305849.474526] Lustre: srv-lustre-OST0002: No data found on store. Initialize space: rc = -61 [305849.497639] Lustre: lustre-OST0002: Imperative Recovery not enabled, recovery window 60-180 [305849.514359] VFS: Open an exclusive opened block device for write dm-4. current [161060 tune2fs]. parent [161059 sh] [305850.349622] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [305852.209076] Lustre: cli-lustre-OST0002-super: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:2:ost] [305859.645110] device-mapper: ioctl: dmsetup[162031]: dm-5 (ost5_flakey) is created successfully [305860.887288] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305860.965871] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [305860.978628] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [305861.003896] Lustre: lustre-OST0004: new disk, initializing [305861.011228] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [305861.034301] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [305861.054753] VFS: Open an exclusive opened block device for write dm-5. current [162341 tune2fs]. parent [162340 sh] [305861.880541] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [305863.329181] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000500000400-0x0000000540000400]:4:ost] [305871.245625] device-mapper: ioctl: dmsetup[163316]: dm-6 (ost7_flakey) is created successfully [305872.478983] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [305872.566323] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [305872.579315] Lustre: lustre-OST0006-osd: enabled 'large_dir' feature on device /dev/mapper/ost7_flakey [305872.603815] Lustre: lustre-OST0006: new disk, initializing [305872.610860] Lustre: srv-lustre-OST0006: No data found on store. Initialize space: rc = -61 [305872.634192] Lustre: lustre-OST0006: Imperative Recovery not enabled, recovery window 60-180 [305872.650894] VFS: Open an exclusive opened block device for write dm-6. current [163627 tune2fs]. parent [163626 sh] [305873.313991] Lustre: cli-lustre-OST0006-super: Allocated super-sequence [0x0000000580000400-0x00000005c0000400]:6:ost] [305873.485676] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [305886.222873] Lustre: DEBUG MARKER: Using TIMEOUT=20 [338535.183138] Lustre: lustre-OST0006: haven't heard from client 7df07f7d-b044-465a-9095-dffb27104d84 (at 192.168.0.82@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 000000007002a456, cur 1709970486 expire 1709970456 last 1709970439 [338536.063514] Lustre: lustre-OST0002: haven't heard from client 7df07f7d-b044-465a-9095-dffb27104d84 (at 192.168.0.82@o2ib) in 47 seconds. I think it's dead, and I am evicting it. exp 000000001158dcbe, cur 1709970486 expire 1709970456 last 1709970439 [338536.088410] Lustre: Skipped 1 previous similar message [338563.581281] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338563.581284] Lustre: lustre-MDT0000-lwp-OST0002: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338563.619311] Lustre: Skipped 1 previous similar message [338573.821174] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338573.840231] Lustre: Skipped 4 previous similar messages [338575.869020] Lustre: 156478:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709970519/real 1709970519] req@000000005969328e x1792997233348160/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1709970526 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [338575.902868] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [338580.989232] Lustre: lustre-MDT0002-lwp-OST0000: Connection to lustre-MDT0002 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338581.008183] Lustre: Skipped 3 previous similar messages [338592.253067] Lustre: lustre-MDT0003-lwp-OST0000: Connection to lustre-MDT0003 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338592.272032] Lustre: Skipped 3 previous similar messages [338602.493008] Lustre: lustre-MDT0004-lwp-OST0000: Connection to lustre-MDT0004 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338602.511904] Lustre: Skipped 3 previous similar messages [338618.876914] Lustre: lustre-MDT0005-lwp-OST0000: Connection to lustre-MDT0005 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338618.896019] Lustre: Skipped 3 previous similar messages [338645.756413] Lustre: 165344:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709970590/real 1709970590] req@000000002fd412bd x1792997233384384/t0(0) o39->lustre-MDT0007-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709970596 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [338645.794625] LustreError: 165344:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [338646.299374] Lustre: server umount lustre-OST0000 complete [338646.524410] Lustre: lustre-MDT0007-lwp-OST0006: Connection to lustre-MDT0007 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [338646.543319] Lustre: Skipped 6 previous similar messages [338648.666660] device-mapper: ioctl: dmsetup[165704]: dm-3 (ost1_flakey) is removed successfully [338651.900344] Lustre: 156468:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1709970595/real 1709970595] req@00000000a9f31c35 x1792997233384448/t0(0) o400->lustre-MDT0007-lwp-OST0002@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1709970602 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [338651.934813] Lustre: 156468:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 5 previous similar messages [338654.176424] LustreError: 165990:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [338654.187780] LustreError: 165990:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [338654.839063] Lustre: server umount lustre-OST0002 complete [338657.219884] device-mapper: ioctl: dmsetup[166351]: dm-4 (ost3_flakey) is removed successfully [338662.861873] LustreError: 166638:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [338662.873313] LustreError: 166638:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [338663.284624] Lustre: server umount lustre-OST0004 complete [338665.724613] device-mapper: ioctl: dmsetup[167000]: dm-5 (ost5_flakey) is removed successfully [338670.984770] LustreError: 167285:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [338670.996087] LustreError: 167285:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [338671.775506] Lustre: server umount lustre-OST0006 complete [338674.202380] device-mapper: ioctl: dmsetup[167647]: dm-6 (ost7_flakey) is removed successfully [338682.220536] Lustre: DEBUG MARKER: server5: executing unload_modules_local [338682.961566] Key type lgssc unregistered [338683.120178] LNet: 168217:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [338684.316070] LNet: Removed LNI 192.168.0.85@o2ib [338684.652220] Key type .llcrypt unregistered [338684.666670] Key type ._llcrypt unregistered [338703.347551] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [338703.358070] alg: No test for adler32 (adler32-zlib) [338704.108060] Key type ._llcrypt registered [338704.113832] Key type .llcrypt registered [338704.137058] Lustre: DEBUG MARKER: server5: executing set_hostid [338708.376297] Lustre: DEBUG MARKER: server5: executing load_modules_local [338708.779021] lnet: unknown parameter '#' ignored [338708.784915] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [338708.792103] lnet: unknown parameter '#' ignored [338708.797886] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [338708.872619] Lustre: Lustre: Build Version: 2.15.4 [338708.936428] LNet: Using FastReg for registration [338709.154044] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [338710.699830] Key type lgssc registered [338710.853578] Lustre: Echo OBD driver; http://www.lustre.org/ [341017.698302] LDISKFS-fs (nvme0n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341048.155396] LDISKFS-fs (nvme1n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341078.558890] LDISKFS-fs (nvme2n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341108.990883] LDISKFS-fs (nvme3n1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341133.741240] Lustre: DEBUG MARKER: server5: executing load_modules_local [341198.086808] device-mapper: ioctl: dmsetup[173350]: dm-3 (ost1_flakey) is created successfully [341199.310142] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341199.407994] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [341199.420764] Lustre: lustre-OST0000-osd: enabled 'large_dir' feature on device /dev/mapper/ost1_flakey [341200.495986] Lustre: lustre-OST0000: new disk, initializing [341200.503224] Lustre: srv-lustre-OST0000: No data found on store. Initialize space: rc = -61 [341200.531436] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [341200.549040] VFS: Open an exclusive opened block device for write dm-3. current [173739 tune2fs]. parent [173738 sh] [341201.373519] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [341203.838886] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000400000400-0x0000000440000400]:0:ost] [341211.696892] device-mapper: ioctl: dmsetup[174715]: dm-4 (ost3_flakey) is created successfully [341212.930830] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341213.011435] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [341213.024181] Lustre: lustre-OST0002-osd: enabled 'large_dir' feature on device /dev/mapper/ost3_flakey [341213.048597] Lustre: lustre-OST0002: new disk, initializing [341213.055947] Lustre: srv-lustre-OST0002: No data found on store. Initialize space: rc = -61 [341213.080008] Lustre: lustre-OST0002: Imperative Recovery not enabled, recovery window 60-180 [341213.100531] VFS: Open an exclusive opened block device for write dm-4. current [175023 tune2fs]. parent [175022 sh] [341213.547839] Lustre: cli-lustre-OST0002-super: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:2:ost] [341213.930077] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [341223.272920] device-mapper: ioctl: dmsetup[175995]: dm-5 (ost5_flakey) is created successfully [341224.511790] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341224.594546] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [341224.607815] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [341224.633056] Lustre: lustre-OST0004: new disk, initializing [341224.640123] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [341224.663848] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [341224.684004] VFS: Open an exclusive opened block device for write dm-5. current [176307 tune2fs]. parent [176306 sh] [341224.699043] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000500000400-0x0000000540000400]:4:ost] [341225.516336] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [341234.827069] device-mapper: ioctl: dmsetup[177283]: dm-6 (ost7_flakey) is created successfully [341236.064321] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [341236.152042] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [341236.164987] Lustre: lustre-OST0006-osd: enabled 'large_dir' feature on device /dev/mapper/ost7_flakey [341236.189999] Lustre: lustre-OST0006: new disk, initializing [341236.197040] Lustre: srv-lustre-OST0006: No data found on store. Initialize space: rc = -61 [341236.220341] Lustre: lustre-OST0006: Imperative Recovery not enabled, recovery window 60-180 [341236.240587] VFS: Open an exclusive opened block device for write dm-6. current [177594 tune2fs]. parent [177593 sh] [341237.076711] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [341237.374880] Lustre: cli-lustre-OST0006-super: Allocated super-sequence [0x0000000580000400-0x00000005c0000400]:6:ost] [341249.832686] Lustre: DEBUG MARKER: Using TIMEOUT=20 [345875.425328] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [345876.469009] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [385423.192171] Lustre: ll_ost01_014: service thread pid 179424 was inactive for 40.035 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [385423.192173] Pid: 179435, comm: ll_ost01_025 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [385423.192177] Call Trace TBD: [385423.192179] Lustre: ll_ost01_032: service thread pid 179442 was inactive for 40.064 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [385423.213646] Lustre: Skipped 2 previous similar messages [385423.225999] Pid: 174283, comm: ll_ost01_006 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [385423.226001] Call Trace TBD: [385423.270360] Pid: 179424, comm: ll_ost01_014 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [385423.282588] Call Trace TBD: [385425.240158] Lustre: ll_ost00_024: service thread pid 179401 was inactive for 41.958 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [385425.240160] Lustre: ll_ost00_034: service thread pid 179448 was inactive for 41.958 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [385425.240161] Lustre: ll_ost00_023: service thread pid 179400 was inactive for 41.958 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [385425.240163] Lustre: ll_ost00_017: service thread pid 179394 was inactive for 41.958 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [385425.240168] Lustre: Skipped 7 previous similar messages [385425.240169] Lustre: Skipped 7 previous similar messages [385425.240170] Lustre: Skipped 7 previous similar messages [385429.353252] Lustre: lustre-OST0000: haven't heard from client 3826deae-7f09-47e2-a191-b02b1b89c7b5 (at 192.168.0.82@o2ib) in 49 seconds. I think it's dead, and I am evicting it. exp 0000000058123c43, cur 1710017380 expire 1710017350 last 1710017331 [385429.378455] LustreError: 179448:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000066d4e049 x1793034539853760/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385430.186988] LustreError: 179394:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000310a06a5 x1793034539955840/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385430.211909] LustreError: 179394:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 13 previous similar messages [385431.213542] LustreError: 173674:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000001acf82b7 x1793034540059136/t0(0) o104->lustre-OST0000@192.168.0.82@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385431.238552] LustreError: 173674:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 1 previous similar message [385433.222975] LustreError: 179410:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000008b67fcf x1793034540555712/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385433.248034] LustreError: 179410:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 7 previous similar messages [385434.097845] Lustre: lustre-OST0002: haven't heard from client 3826deae-7f09-47e2-a191-b02b1b89c7b5 (at 192.168.0.82@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 000000003c0a3664, cur 1710017385 expire 1710017355 last 1710017337 [385434.123041] Lustre: Skipped 1 previous similar message [385436.886792] Lustre: lustre-OST0004: haven't heard from client 3826deae-7f09-47e2-a191-b02b1b89c7b5 (at 192.168.0.82@o2ib) in 48 seconds. I think it's dead, and I am evicting it. exp 00000000af6335b7, cur 1710017388 expire 1710017358 last 1710017340 [385436.911965] Lustre: Skipped 1 previous similar message [385437.296067] LustreError: 179401:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000a42bdbd8 x1793034540810688/t0(0) o104->lustre-OST0000@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385437.321176] LustreError: 179401:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 36 previous similar messages [385445.371948] LustreError: 179409:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000d7992577 x1793034549364928/t0(0) o104->lustre-OST0004@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385445.397054] LustreError: 179409:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 109237 previous similar messages [385462.547817] LustreError: 179399:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000004d973c7e x1793034549374784/t0(0) o104->lustre-OST0006@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385462.572916] LustreError: 179399:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 22 previous similar messages [385494.589008] LustreError: 179405:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000dd0cff2c x1793034549395520/t0(0) o104->lustre-OST0006@192.168.0.81@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [385494.614081] LustreError: 179405:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 124 previous similar messages [385845.332454] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [385845.351524] Lustre: Skipped 2 previous similar messages [385855.572349] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [385855.591394] Lustre: Skipped 4 previous similar messages [385857.620132] Lustre: 170431:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1710017801/real 1710017801] req@0000000018874167 x1793034549542656/t0(0) o400->MGC192.168.0.83@o2ib@192.168.0.83@o2ib:26/25 lens 224/224 e 0 to 1 dl 1710017808 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [385857.653962] LustreError: 166-1: MGC192.168.0.83@o2ib: Connection to MGS (at 192.168.0.83@o2ib) was lost; in progress operations using this service will fail [385862.740361] Lustre: lustre-MDT0002-lwp-OST0002: Connection to lustre-MDT0002 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [385862.759370] Lustre: Skipped 3 previous similar messages [385874.004283] Lustre: lustre-MDT0003-lwp-OST0002: Connection to lustre-MDT0003 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [385874.023513] Lustre: Skipped 3 previous similar messages [385885.268079] Lustre: lustre-MDT0004-lwp-OST0000: Connection to lustre-MDT0004 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [385885.287111] Lustre: Skipped 3 previous similar messages [385901.652005] Lustre: lustre-MDT0006-lwp-OST0000: Connection to lustre-MDT0006 (at 192.168.0.83@o2ib) was lost; in progress operations using this service will wait for recovery to complete [385901.671122] Lustre: Skipped 7 previous similar messages [385919.059571] Lustre: 170477:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1710017863/real 1710017863] req@000000006ba31b05 x1793034549576000/t0(0) o400->lustre-MDT0007-lwp-OST0004@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1710017870 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:2.0' [385919.059585] Lustre: lustre-MDT0007-lwp-OST0006: Connection to lustre-MDT0007 (at 192.168.0.84@o2ib) was lost; in progress operations using this service will wait for recovery to complete [385919.093920] Lustre: 170477:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 2 previous similar messages [385919.112664] Lustre: Skipped 1 previous similar message [385920.851547] Lustre: 179689:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1710017866/real 1710017866] req@00000000fec9e3dc x1793034549577088/t0(0) o39->lustre-MDT0007-lwp-OST0000@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1710017872 ref 2 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'umount.0' [385920.889676] LustreError: 179689:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [385921.213247] Lustre: server umount lustre-OST0000 complete [385923.374910] device-mapper: ioctl: dmsetup[180050]: dm-3 (ost1_flakey) is removed successfully [385924.947513] Lustre: 170409:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1710017869/real 1710017869] req@00000000d92fc5ab x1793034549577152/t0(0) o400->lustre-MDT0007-lwp-OST0002@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1710017876 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [385924.947516] Lustre: 170411:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1710017869/real 1710017869] req@000000008d3eb07a x1793034549577280/t0(0) o400->lustre-MDT0007-lwp-OST0006@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1710017876 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [385924.947519] Lustre: 170410:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1710017869/real 1710017869] req@00000000c68197fa x1793034549577216/t0(0) o400->lustre-MDT0007-lwp-OST0004@192.168.0.84@o2ib:12/10 lens 224/224 e 0 to 1 dl 1710017876 ref 1 fl Rpc:XNQr/0/ffffffff rc 0/-1 job:'kworker/u192:0.0' [385924.947524] Lustre: 170411:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [385924.947526] Lustre: 170410:0:(client.c:2289:ptlrpc_expire_one_request()) Skipped 3 previous similar messages [385928.182366] LustreError: 180337:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [385928.193669] LustreError: 180337:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [385928.330031] Lustre: server umount lustre-OST0002 complete [385930.179067] device-mapper: ioctl: dmsetup[180698]: dm-4 (ost3_flakey) is removed successfully [385934.718206] LustreError: 180984:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [385934.729384] LustreError: 180984:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [385934.892000] Lustre: server umount lustre-OST0004 complete [385936.800577] device-mapper: ioctl: dmsetup[181345]: dm-5 (ost5_flakey) is removed successfully [385941.174760] LustreError: 181629:0:(lprocfs_jobstats.c:137:job_stat_exit()) should not have any items [385941.186168] LustreError: 181629:0:(lprocfs_jobstats.c:137:job_stat_exit()) Skipped 1 previous similar message [385941.353696] Lustre: server umount lustre-OST0006 complete [385943.306421] device-mapper: ioctl: dmsetup[181989]: dm-6 (ost7_flakey) is removed successfully [385950.486266] Lustre: DEBUG MARKER: server5: executing unload_modules_local [385951.227701] Key type lgssc unregistered [385951.427350] LNet: 182560:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [385952.627253] LNet: Removed LNI 192.168.0.85@o2ib [385953.015538] Key type .llcrypt unregistered [385953.029905] Key type ._llcrypt unregistered [391192.655094] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [391193.018280] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [431462.821816] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [431464.297819] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [498605.593173] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [498607.465590] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [567091.088504] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [567092.872519] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [574417.083271] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [574417.511345] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [652245.091805] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [652246.326150] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [659026.285400] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [659027.189766] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [690251.741645] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [690252.121681] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1040369.136674] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1040370.279485] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1201087.119447] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1201087.550641] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1252834.741448] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1252836.229659] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1325435.585936] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1325436.652409] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1510538.661981] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1510539.855069] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1510820.716016] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1510821.292467] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1518849.602456] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1518850.658263] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1527785.016849] ixgbe 0000:01:00.0 enp1s0f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1527785.035227] IPv6: ADDRCONF(NETDEV_CHANGE): enp1s0f0: link becomes ready [1527787.015943] ixgbe 0000:01:00.0 enp1s0f0: NIC Link is Down [1538668.084715] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1538669.067946] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1543073.794474] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1543075.395323] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1585009.638800] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1585010.623105] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1609983.185576] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1609983.992285] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1610708.881462] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1610709.265576] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1615117.285184] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1615118.152831] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1629129.534672] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1629130.951239] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1645040.402264] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1645041.972305] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1694533.657834] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1694534.443419] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1742222.639082] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1742224.533816] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [1791823.419686] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [1791824.209917] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2040549.205994] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2040550.424756] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2042837.959496] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2042839.684288] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2054389.606181] capability: warning: `dnf' uses 32-bit capabilities (legacy support in use) [2129944.252951] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2129944.789763] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2132269.917956] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2132271.041037] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2155843.150334] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2155845.134460] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2206264.450239] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2206265.742050] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2209871.508592] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2209873.454228] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2256295.988697] systemd-rc-local-generator[217537]: /etc/rc.d/rc.local is not marked executable, skipping. [2256295.989201] systemd-sysv-generator[217541]: SysV service '/etc/rc.d/init.d/lsvcgss' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2256296.027466] systemd-sysv-generator[217541]: SysV service '/etc/rc.d/init.d/lustre' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2256296.058367] systemd-sysv-generator[217541]: SysV service '/etc/rc.d/init.d/mst' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2256308.539726] systemd-rc-local-generator[218098]: /etc/rc.d/rc.local is not marked executable, skipping. [2256308.540678] systemd-sysv-generator[218102]: SysV service '/etc/rc.d/init.d/lsvcgss' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2256308.579133] systemd-sysv-generator[218102]: SysV service '/etc/rc.d/init.d/lustre' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2256308.610205] systemd-sysv-generator[218102]: SysV service '/etc/rc.d/init.d/mst' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2421899.898966] nvme0n1: [2421900.032252] nvme1n1: p1 [2421900.142292] nvme2n1: p1 [2423329.513131] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [2423329.523969] alg: No test for adler32 (adler32-zlib) [2423330.273117] Key type ._llcrypt registered [2423330.278762] Key type .llcrypt registered [2423330.295316] lnet: unknown parameter '#' ignored [2423330.301241] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [2423330.325251] Lustre: Lustre: Build Version: 2.15.4 [2423330.375294] LNet: Using FastReg for registration [2423330.576240] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [2423831.039480] systemd-rc-local-generator[242293]: /etc/rc.d/rc.local is not marked executable, skipping. [2423831.042422] systemd-sysv-generator[242297]: SysV service '/etc/rc.d/init.d/mst' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2424142.022275] systemd-rc-local-generator[290692]: /etc/rc.d/rc.local is not marked executable, skipping. [2424142.023971] systemd-sysv-generator[290696]: SysV service '/etc/rc.d/init.d/lsvcgss' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2424142.061338] systemd-sysv-generator[290696]: SysV service '/etc/rc.d/init.d/lustre' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2424142.089373] systemd-sysv-generator[290696]: SysV service '/etc/rc.d/init.d/mst' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust. [2428916.966926] LNet: 291412:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [2428918.138838] LNet: Removed LNI 192.168.0.85@o2ib [2428918.382502] Key type .llcrypt unregistered [2428918.397062] Key type ._llcrypt unregistered [2428918.472671] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [2428918.483544] alg: No test for adler32 (adler32-zlib) [2428919.230908] Key type ._llcrypt registered [2428919.236561] Key type .llcrypt registered [2428919.253142] lnet: unknown parameter '#' ignored [2428919.259072] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [2428919.283032] Lustre: Lustre: Build Version: 2.15.4 [2428919.333705] LNet: Using FastReg for registration [2428919.536048] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [2429766.544328] VFS: Open an exclusive opened block device for write nvme0n1. current [292313 wipefs]. parent [292312 parted_disks.sh] [2429766.577743] nvme0n1: p1 [2429766.676422] VFS: Open an exclusive opened block device for write nvme1n1. current [292326 wipefs]. parent [292312 parted_disks.sh] [2429766.802607] VFS: Open an exclusive opened block device for write nvme2n1. current [292336 wipefs]. parent [292312 parted_disks.sh] [2429766.837919] nvme2n1: p1 [2429766.932275] VFS: Open an exclusive opened block device for write nvme3n1. current [292351 wipefs]. parent [292312 parted_disks.sh] [2429766.965284] nvme3n1: p1 [2429945.217710] LNet: 292995:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [2429946.385617] LNet: Removed LNI 192.168.0.85@o2ib [2429946.625053] Key type .llcrypt unregistered [2429946.639449] Key type ._llcrypt unregistered [2429946.731357] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [2429946.741144] alg: No test for adler32 (adler32-zlib) [2429947.489725] Key type ._llcrypt registered [2429947.495064] Key type .llcrypt registered [2429947.511533] lnet: unknown parameter '#' ignored [2429947.517381] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [2429947.541066] Lustre: Lustre: Build Version: 2.15.4 [2429947.590711] LNet: Using FastReg for registration [2429947.790009] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [2430071.188309] Lustre: DEBUG MARKER: server5: executing set_hostid [2430074.951100] Lustre: DEBUG MARKER: server5: executing load_modules_local [2430076.896433] Key type lgssc registered [2430076.990486] Lustre: Echo OBD driver; http://www.lustre.org/ [2430396.254040] Lustre: DEBUG MARKER: server5: executing unload_modules_local [2430396.746517] Key type lgssc unregistered [2430396.861689] LNet: 304609:0:(lib-ptl.c:958:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [2430398.029574] LNet: Removed LNI 192.168.0.85@o2ib [2430398.249787] Key type .llcrypt unregistered [2430398.264352] Key type ._llcrypt unregistered [2430554.953311] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [2430554.963593] alg: No test for adler32 (adler32-zlib) [2430555.712292] Key type ._llcrypt registered [2430555.717607] Key type .llcrypt registered [2430555.740652] Lustre: DEBUG MARKER: server5: executing set_hostid [2430560.468999] Lustre: DEBUG MARKER: server5: executing load_modules_local [2430560.892091] lnet: unknown parameter '#' ignored [2430560.898000] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [2430560.972241] Lustre: Lustre: Build Version: 2.15.4 [2430561.031038] LNetError: 309982:0:(o2iblnd.c:3327:kiblnd_startup()) ko2iblnd: No matching interfaces [2430562.060113] LNetError: 105-4: Error -100 starting up LNI o2ib [2430562.068010] LustreError: 309982:0:(events.c:639:ptlrpc_init_portals()) network initialisation failed [2473440.140233] Key type .llcrypt unregistered [2473440.154712] Key type ._llcrypt unregistered [2473440.238267] LNet: HW NUMA nodes: 4, HW CPU cores: 96, npartitions: 4 [2473440.248180] alg: No test for adler32 (adler32-zlib) [2473440.996731] Key type ._llcrypt registered [2473441.002224] Key type .llcrypt registered [2473441.018674] lnet: unknown parameter '#' ignored [2473441.024519] lnet: unknown parameter 'o2ib1(ibp129s0f1)' ignored [2473441.048328] Lustre: Lustre: Build Version: 2.15.4 [2473441.098949] LNet: Using FastReg for registration [2473441.309975] LNet: Added LNI 192.168.0.85@o2ib [8/256/0/180] [2473577.382776] Lustre: DEBUG MARKER: server5: executing set_hostid [2473581.175552] Lustre: DEBUG MARKER: server5: executing load_modules_local [2473583.143350] Key type lgssc registered [2473583.232585] Lustre: Echo OBD driver; http://www.lustre.org/ [2473905.499762] LDISKFS-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2474285.742108] LDISKFS-fs (nvme1n1p1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2474669.001201] LDISKFS-fs (nvme2n1p1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475052.367961] LDISKFS-fs (nvme3n1p1): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475143.306182] LDISKFS-fs (nvme0n1p2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475176.940024] LDISKFS-fs (nvme1n1p2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475210.610422] LDISKFS-fs (nvme2n1p2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475246.146332] LDISKFS-fs (nvme3n1p2): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475278.252926] LDISKFS-fs (nvme0n1p3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475311.639974] LDISKFS-fs (nvme1n1p3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475345.125493] LDISKFS-fs (nvme2n1p3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475380.708939] LDISKFS-fs (nvme3n1p3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475412.852774] LDISKFS-fs (nvme0n1p4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475446.293842] LDISKFS-fs (nvme1n1p4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475479.819698] LDISKFS-fs (nvme2n1p4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475515.485786] LDISKFS-fs (nvme3n1p4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475547.703330] LDISKFS-fs (nvme0n1p5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475581.324051] LDISKFS-fs (nvme1n1p5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475615.043216] LDISKFS-fs (nvme2n1p5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475649.528936] LDISKFS-fs (nvme3n1p5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475696.904052] Lustre: DEBUG MARKER: server5: executing load_modules_local [2475730.198695] device-mapper: ioctl: dmsetup[328031]: dm-3 (mds5_flakey) is created successfully [2475731.514941] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475731.732989] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [2475731.835156] Lustre: srv-lustre-MDT0004: No data found on store. Initialize space: rc = -61 [2475732.333471] Lustre: lustre-MDT0004: Not available for connect from 192.168.0.83@o2ib (not set up) [2475732.853685] Lustre: lustre-MDT0004: new disk, initializing [2475732.880306] Lustre: lustre-MDT0004: Imperative Recovery not enabled, recovery window 60-180 [2475732.893135] Lustre: cli-ctl-lustre-MDT0004: Allocated super-sequence [0x0000000300000400-0x0000000340000400]:4:mdt] [2475732.900427] VFS: Open an exclusive opened block device for write dm-3. current [328429 tune2fs]. parent [328428 sh] [2475733.701841] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475770.642626] device-mapper: ioctl: dmsetup[329420]: dm-4 (mds11_flakey) is created successfully [2475771.972946] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475772.182109] LDISKFS-fs (dm-4): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [2475772.210339] Lustre: srv-lustre-MDT000a: No data found on store. Initialize space: rc = -61 [2475772.224484] Lustre: lustre-MDT000a: Not available for connect from 192.168.0.82@o2ib (not set up) [2475772.235648] Lustre: Skipped 2 previous similar messages [2475772.240348] Lustre: lustre-MDT000a: new disk, initializing [2475772.265072] Lustre: lustre-MDT000a: Imperative Recovery not enabled, recovery window 60-180 [2475772.281535] Lustre: cli-ctl-lustre-MDT000a: Allocated super-sequence [0x0000000480000400-0x00000004c0000400]:a:mdt] [2475772.289233] VFS: Open an exclusive opened block device for write dm-4. current [329757 tune2fs]. parent [329756 sh] [2475773.098141] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475801.533476] device-mapper: ioctl: dmsetup[330699]: dm-5 (mds17_flakey) is created successfully [2475802.870646] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475803.076503] LDISKFS-fs (dm-5): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [2475803.108565] Lustre: srv-lustre-MDT0010: No data found on store. Initialize space: rc = -61 [2475803.139831] Lustre: lustre-MDT0010: new disk, initializing [2475803.172105] Lustre: lustre-MDT0010: Imperative Recovery not enabled, recovery window 60-180 [2475803.185165] Lustre: cli-ctl-lustre-MDT0010: Allocated super-sequence [0x0000000600000400-0x0000000640000400]:10:mdt] [2475803.193498] VFS: Open an exclusive opened block device for write dm-5. current [331052 tune2fs]. parent [331051 sh] [2475804.019218] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475840.483222] device-mapper: ioctl: dmsetup[332107]: dm-6 (mds23_flakey) is created successfully [2475841.836468] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475842.044216] LDISKFS-fs (dm-6): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [2475842.081042] Lustre: srv-lustre-MDT0016: No data found on store. Initialize space: rc = -61 [2475842.111988] Lustre: lustre-MDT0016: new disk, initializing [2475842.149764] Lustre: lustre-MDT0016: Imperative Recovery not enabled, recovery window 60-180 [2475842.164898] Lustre: cli-ctl-lustre-MDT0016: Allocated super-sequence [0x0000000780000400-0x00000007c0000400]:16:mdt] [2475842.173214] VFS: Open an exclusive opened block device for write dm-6. current [332480 tune2fs]. parent [332479 sh] [2475843.014793] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475879.044451] device-mapper: ioctl: dmsetup[333539]: dm-7 (ost5_flakey) is created successfully [2475880.288764] LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475880.348486] LDISKFS-fs (dm-7): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2475880.361258] Lustre: lustre-OST0004-osd: enabled 'large_dir' feature on device /dev/mapper/ost5_flakey [2475880.434376] Lustre: lustre-OST0004: new disk, initializing [2475880.441669] Lustre: srv-lustre-OST0004: No data found on store. Initialize space: rc = -61 [2475880.478664] Lustre: lustre-OST0004: Imperative Recovery not enabled, recovery window 60-180 [2475880.497950] VFS: Open an exclusive opened block device for write dm-7. current [333918 tune2fs]. parent [333917 sh] [2475880.603316] Lustre: cli-lustre-OST0004-super: Allocated super-sequence [0x0000000900000400-0x0000000940000400]:4:ost] [2475881.335933] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475917.287056] device-mapper: ioctl: dmsetup[335005]: dm-8 (ost11_flakey) is created successfully [2475918.540797] LDISKFS-fs (dm-8): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475918.598240] LDISKFS-fs (dm-8): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2475918.610963] Lustre: lustre-OST000a-osd: enabled 'large_dir' feature on device /dev/mapper/ost11_flakey [2475918.642524] Lustre: lustre-OST000a: new disk, initializing [2475918.650007] Lustre: srv-lustre-OST000a: No data found on store. Initialize space: rc = -61 [2475918.687136] Lustre: lustre-OST000a: Imperative Recovery not enabled, recovery window 60-180 [2475918.708846] VFS: Open an exclusive opened block device for write dm-8. current [335332 tune2fs]. parent [335331 sh] [2475919.557851] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475919.940929] Lustre: cli-lustre-OST000a-super: Allocated super-sequence [0x0000000a80000400-0x0000000ac0000400]:a:ost] [2475947.892410] device-mapper: ioctl: dmsetup[336302]: dm-9 (ost17_flakey) is created successfully [2475949.155260] LDISKFS-fs (dm-9): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475949.214726] LDISKFS-fs (dm-9): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2475949.228037] Lustre: lustre-OST0010-osd: enabled 'large_dir' feature on device /dev/mapper/ost17_flakey [2475949.260044] Lustre: lustre-OST0010: new disk, initializing [2475949.267168] Lustre: srv-lustre-OST0010: No data found on store. Initialize space: rc = -61 [2475949.314717] Lustre: lustre-OST0010: Imperative Recovery not enabled, recovery window 60-180 [2475949.332666] VFS: Open an exclusive opened block device for write dm-9. current [336629 tune2fs]. parent [336628 sh] [2475949.369548] Lustre: cli-lustre-OST0010-super: Allocated super-sequence [0x0000000c00000400-0x0000000c40000400]:10:ost] [2475950.194699] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475986.919749] device-mapper: ioctl: dmsetup[337717]: dm-10 (ost23_flakey) is created successfully [2475988.196710] LDISKFS-fs (dm-10): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2475988.255519] LDISKFS-fs (dm-10): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2475988.268661] Lustre: lustre-OST0016-osd: enabled 'large_dir' feature on device /dev/mapper/ost23_flakey [2475988.302141] Lustre: lustre-OST0016: new disk, initializing [2475988.304161] Lustre: lustre-OST0016: Not available for connect from 192.168.0.82@o2ib (not set up) [2475988.309572] Lustre: srv-lustre-OST0016: No data found on store. Initialize space: rc = -61 [2475988.320641] Lustre: Skipped 3 previous similar messages [2475988.367777] Lustre: lustre-OST0016: Imperative Recovery not enabled, recovery window 60-180 [2475988.388310] VFS: Open an exclusive opened block device for write dm-10. current [338067 tune2fs]. parent [338066 sh] [2475989.260260] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2475990.301207] Lustre: cli-lustre-OST0016-super: Allocated super-sequence [0x0000000d80000400-0x0000000dc0000400]:16:ost] [2476023.379136] device-mapper: ioctl: dmsetup[339144]: dm-11 (ost29_flakey) is created successfully [2476024.672796] LDISKFS-fs (dm-11): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476024.735258] LDISKFS-fs (dm-11): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476024.748384] Lustre: lustre-OST001c-osd: enabled 'large_dir' feature on device /dev/mapper/ost29_flakey [2476024.816176] VFS: Open an exclusive opened block device for write dm-11. current [339469 tune2fs]. parent [339468 sh] [2476025.694191] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476057.655360] device-mapper: ioctl: dmsetup[340502]: dm-12 (ost35_flakey) is created successfully [2476058.955734] LDISKFS-fs (dm-12): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476059.024765] LDISKFS-fs (dm-12): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476059.037574] Lustre: lustre-OST0022-osd: enabled 'large_dir' feature on device /dev/mapper/ost35_flakey [2476059.072100] Lustre: lustre-OST0022: new disk, initializing [2476059.079627] Lustre: Skipped 1 previous similar message [2476059.089025] Lustre: srv-lustre-OST0022: No data found on store. Initialize space: rc = -61 [2476059.099154] Lustre: Skipped 1 previous similar message [2476059.133642] Lustre: lustre-OST0022: Imperative Recovery not enabled, recovery window 60-180 [2476059.144117] Lustre: Skipped 1 previous similar message [2476059.156669] Lustre: cli-lustre-OST0022-super: Allocated super-sequence [0x0000001080000400-0x00000010c0000400]:22:ost] [2476059.158907] VFS: Open an exclusive opened block device for write dm-12. current [340829 tune2fs]. parent [340828 sh] [2476059.169613] Lustre: Skipped 1 previous similar message [2476060.045901] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476097.385570] device-mapper: ioctl: dmsetup[341938]: dm-13 (ost41_flakey) is created successfully [2476098.704095] LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476098.766417] LDISKFS-fs (dm-13): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476098.779447] Lustre: lustre-OST0028-osd: enabled 'large_dir' feature on device /dev/mapper/ost41_flakey [2476098.851189] VFS: Open an exclusive opened block device for write dm-13. current [342262 tune2fs]. parent [342261 sh] [2476099.739429] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476131.916761] device-mapper: ioctl: dmsetup[343343]: dm-14 (ost47_flakey) is created successfully [2476133.245872] LDISKFS-fs (dm-14): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476133.304400] LDISKFS-fs (dm-14): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476133.317180] Lustre: lustre-OST002e-osd: enabled 'large_dir' feature on device /dev/mapper/ost47_flakey [2476133.383910] Lustre: lustre-OST002e: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476133.406463] VFS: Open an exclusive opened block device for write dm-14. current [343706 tune2fs]. parent [343705 sh] [2476134.301584] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476169.521628] device-mapper: ioctl: dmsetup[344743]: dm-15 (ost53_flakey) is created successfully [2476170.850797] LDISKFS-fs (dm-15): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476170.908422] LDISKFS-fs (dm-15): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476170.984572] Lustre: lustre-OST0034: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476171.006486] VFS: Open an exclusive opened block device for write dm-15. current [345069 tune2fs]. parent [345068 sh] [2476171.897917] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476210.242463] device-mapper: ioctl: dmsetup[346204]: dm-16 (ost59_flakey) is created successfully [2476211.586383] LDISKFS-fs (dm-16): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476211.647600] LDISKFS-fs (dm-16): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476211.660432] Lustre: lustre-OST003a-osd: enabled 'large_dir' feature on device /dev/mapper/ost59_flakey [2476211.671617] Lustre: Skipped 1 previous similar message [2476211.710280] Lustre: lustre-OST003a: new disk, initializing [2476211.717767] Lustre: Skipped 3 previous similar messages [2476211.727651] Lustre: srv-lustre-OST003a: No data found on store. Initialize space: rc = -61 [2476211.737771] Lustre: Skipped 3 previous similar messages [2476211.774403] Lustre: lustre-OST003a: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476211.797409] VFS: Open an exclusive opened block device for write dm-16. current [346530 tune2fs]. parent [346529 sh] [2476212.702691] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476213.574405] Lustre: cli-lustre-OST003a-super: Allocated super-sequence [0x0000001680000400-0x00000016c0000400]:3a:ost] [2476213.587295] Lustre: Skipped 3 previous similar messages [2476243.274485] device-mapper: ioctl: dmsetup[347576]: dm-17 (ost65_flakey) is created successfully [2476244.640774] LDISKFS-fs (dm-17): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476244.701311] LDISKFS-fs (dm-17): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476244.777431] Lustre: lustre-OST0040: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476244.799111] VFS: Open an exclusive opened block device for write dm-17. current [347937 tune2fs]. parent [347936 sh] [2476245.716432] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476284.538222] device-mapper: ioctl: dmsetup[349065]: dm-18 (ost71_flakey) is created successfully [2476285.896534] LDISKFS-fs (dm-18): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476285.962974] LDISKFS-fs (dm-18): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476286.046749] Lustre: lustre-OST0046: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476286.069650] VFS: Open an exclusive opened block device for write dm-18. current [349391 tune2fs]. parent [349390 sh] [2476286.988293] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476323.343999] device-mapper: ioctl: dmsetup[350490]: dm-19 (ost77_flakey) is created successfully [2476324.711331] LDISKFS-fs (dm-19): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476324.782403] LDISKFS-fs (dm-19): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476324.869595] Lustre: lustre-OST004c: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476324.892602] VFS: Open an exclusive opened block device for write dm-19. current [350854 tune2fs]. parent [350853 sh] [2476325.815048] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476359.814346] device-mapper: ioctl: dmsetup[351925]: dm-20 (ost83_flakey) is created successfully [2476361.201732] LDISKFS-fs (dm-20): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476361.264078] LDISKFS-fs (dm-20): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476361.277244] Lustre: lustre-OST0052-osd: enabled 'large_dir' feature on device /dev/mapper/ost83_flakey [2476361.288512] Lustre: Skipped 3 previous similar messages [2476361.370537] Lustre: lustre-OST0052: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476361.393375] VFS: Open an exclusive opened block device for write dm-20. current [352250 tune2fs]. parent [352249 sh] [2476362.334465] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476402.020367] device-mapper: ioctl: dmsetup[353396]: dm-21 (ost89_flakey) is created successfully [2476403.424029] LDISKFS-fs (dm-21): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476403.489826] LDISKFS-fs (dm-21): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476403.576866] Lustre: lustre-OST0058: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [2476403.600414] VFS: Open an exclusive opened block device for write dm-21. current [353765 tune2fs]. parent [353764 sh] [2476404.531998] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476438.994444] device-mapper: ioctl: dmsetup[354926]: dm-22 (ost95_flakey) is created successfully [2476440.405868] LDISKFS-fs (dm-22): mounted filesystem with ordered data mode. Opts: errors=remount-ro [2476440.465678] LDISKFS-fs (dm-22): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [2476440.554789] VFS: Open an exclusive opened block device for write dm-22. current [355254 tune2fs]. parent [355253 sh] [2476441.509878] Lustre: DEBUG MARKER: server5: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 192 [2476453.523097] Lustre: DEBUG MARKER: Using TIMEOUT=20 [2476466.746686] Lustre: DEBUG MARKER: server1: executing check_logdir /tmp/test_logs/1712108434 [2476467.579920] Lustre: DEBUG MARKER: client1: executing check_logdir /tmp/test_logs/1712108434 [2476469.594661] Lustre: DEBUG MARKER: server1: executing yml_node [2476470.384694] Lustre: DEBUG MARKER: client1: executing yml_node [2476473.424995] Lustre: DEBUG MARKER: Client: 2.15.4 [2476474.500346] Lustre: DEBUG MARKER: MDS: 2.15.4 [2476475.549982] Lustre: DEBUG MARKER: OSS: 2.15.4 [2482145.852563] Lustre: DEBUG MARKER: server1: executing check_logdir /tmp/test_logs/1712114113 [2482146.668872] Lustre: DEBUG MARKER: client1: executing check_logdir /tmp/test_logs/1712114113 [2482148.639813] Lustre: DEBUG MARKER: server1: executing yml_node [2482149.320501] Lustre: DEBUG MARKER: client1: executing yml_node [2482152.598799] Lustre: DEBUG MARKER: Client: 2.15.4 [2482153.690340] Lustre: DEBUG MARKER: MDS: 2.15.4 [2482154.768741] Lustre: DEBUG MARKER: OSS: 2.15.4 [2486449.140650] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX [2486450.442218] ixgbe 0000:01:00.1 enp1s0f1: NIC Link is Down [2488119.942159] Lustre: DEBUG MARKER: server1: executing check_logdir /tmp/test_logs/1712120087 [2488120.826584] Lustre: DEBUG MARKER: client1: executing check_logdir /tmp/test_logs/1712120087 [2488123.038676] Lustre: DEBUG MARKER: server1: executing yml_node [2488124.010815] Lustre: DEBUG MARKER: client1: executing yml_node [2488127.985211] Lustre: DEBUG MARKER: Client: 2.15.4 [2488129.099457] Lustre: DEBUG MARKER: MDS: 2.15.4 [2488130.231372] Lustre: DEBUG MARKER: OSS: 2.15.4 [2494280.918266] Lustre: DEBUG MARKER: server1: executing check_logdir /tmp/test_logs/1712126248 [2494281.819030] Lustre: DEBUG MARKER: client1: executing check_logdir /tmp/test_logs/1712126248 [2494289.034829] Lustre: 361252:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1712126252/real 0] req@000000006053d43f x1795273593793408/t0(0) o104->lustre-OST0058@192.168.0.100@o2ib:15/16 lens 328/224 e 0 to 1 dl 1712126259 ref 2 fl Rpc:Xr/0/ffffffff rc 0/-1 job:'' [2494292.931465] Lustre: DEBUG MARKER: server1: executing yml_node [2494293.772762] Lustre: lustre-OST000a: Client 5d853277-1f18-437e-a94e-e36dde23fe51 (at 192.168.0.100@o2ib) reconnecting [2494293.785276] Lustre: Skipped 7 previous similar messages [2494293.850037] Lustre: DEBUG MARKER: client1: executing yml_node [2494307.939061] Lustre: lustre-OST0004: Client 5d853277-1f18-437e-a94e-e36dde23fe51 (at 192.168.0.100@o2ib) reconnecting [2494307.951460] Lustre: Skipped 6 previous similar messages [2494310.034984] Lustre: lustre-OST000a: Client 5d853277-1f18-437e-a94e-e36dde23fe51 (at 192.168.0.100@o2ib) reconnecting [2494310.047496] Lustre: Skipped 7 previous similar messages [2494321.802518] Lustre: ll_ost01_071: service thread pid 363933 was inactive for 40.062 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494321.802520] Lustre: ll_ost01_037: service thread pid 359919 was inactive for 40.022 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494321.802522] Lustre: ll_ost01_118: service thread pid 368020 was inactive for 40.062 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [2494321.802524] Lustre: ll_ost01_079: service thread pid 363974 was inactive for 40.023 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494321.802527] Lustre: ll_ost01_115: service thread pid 368015 was inactive for 40.062 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494321.802528] Lustre: ll_ost01_107: service thread pid 368000 was inactive for 40.023 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494321.802530] Lustre: ll_ost01_051: service thread pid 363886 was inactive for 40.063 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [2494321.802532] Lustre: ll_ost01_112: service thread pid 368009 was inactive for 40.063 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494321.802534] Lustre: ll_ost01_108: service thread pid 368002 was inactive for 40.023 seconds. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [2494321.802539] Lustre: Skipped 7 previous similar messages [2494321.802541] Pid: 368020, comm: ll_ost01_118 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [2494321.802542] Lustre: Skipped 11 previous similar messages [2494321.802543] Lustre: Skipped 11 previous similar messages [2494321.802545] Lustre: Skipped 11 previous similar messages [2494321.802546] Lustre: Skipped 11 previous similar messages [2494321.802547] Call Trace TBD: [2494321.802549] Pid: 363886, comm: ll_ost01_051 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [2494321.802550] Call Trace TBD: [2494321.802552] Pid: 368002, comm: ll_ost01_108 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [2494321.802553] Call Trace TBD: [2494321.819793] Lustre: Skipped 6 previous similar messages [2494323.850501] Lustre: ll_ost01_011: service thread pid 359856 was inactive for 41.991 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494323.850504] Lustre: ll_ost01_098: service thread pid 367987 was inactive for 42.025 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494323.850506] Lustre: ll_ost01_099: service thread pid 367988 was inactive for 42.025 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494323.850509] Lustre: ll_ost01_101: service thread pid 367990 was inactive for 42.022 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494323.850513] Lustre: ll_ost01_104: service thread pid 367994 was inactive for 42.026 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494323.850519] Lustre: Skipped 4 previous similar messages [2494323.850520] Lustre: Skipped 4 previous similar messages [2494323.850522] Lustre: Skipped 4 previous similar messages [2494323.850524] Lustre: Skipped 4 previous similar messages [2494323.867730] Lustre: Skipped 10 previous similar messages [2494331.114850] Lustre: lustre-OST0004: Client 5d853277-1f18-437e-a94e-e36dde23fe51 (at 192.168.0.100@o2ib) reconnecting [2494331.127295] Lustre: Skipped 5 previous similar messages [2494331.530440] Lustre: 361252:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1712126285/real 1712126285] req@000000006053d43f x1795273593793408/t0(0) o104->lustre-OST0058@192.168.0.100@o2ib:15/16 lens 328/224 e 0 to 1 dl 1712126302 ref 2 fl Rpc:Xr/2/ffffffff rc -11/-1 job:'' [2494335.525965] Lustre: lustre-OST005e: Client 5d853277-1f18-437e-a94e-e36dde23fe51 (at 192.168.0.100@o2ib) reconnecting [2494346.518718] Lustre: lustre-OST0034: Client 5d853277-1f18-437e-a94e-e36dde23fe51 (at 192.168.0.100@o2ib) reconnecting [2494346.531230] Lustre: Skipped 8 previous similar messages [2494350.474259] Lustre: ll_ost01_074: service thread pid 363940 was inactive for 63.357 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494350.474262] Lustre: ll_ost01_096: service thread pid 367985 was inactive for 63.357 seconds. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [2494350.474268] Lustre: Skipped 2 previous similar messages [2494350.491611] Lustre: Skipped 8 previous similar messages [2494367.309952] Lustre: lustre-OST0004: Client 5d853277-1f18-437e-a94e-e36dde23fe51 (at 192.168.0.100@o2ib) reconnecting [2494367.322453] Lustre: Skipped 5 previous similar messages [2494368.906096] Lustre: 361252:0:(client.c:2289:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1712126322/real 1712126322] req@000000006053d43f x1795273593793408/t0(0) o104->lustre-OST0058@192.168.0.100@o2ib:15/16 lens 328/224 e 0 to 1 dl 1712126339 ref 2 fl Rpc:Xr/2/ffffffff rc -11/-1 job:'' [2494380.726853] Lustre: DEBUG MARKER: Client: 2.15.4 [2494381.961776] Lustre: DEBUG MARKER: MDS: 2.15.4 [2494382.918178] LustreError: 328350:0:(ldlm_lockd.c:257:expired_lock_main()) ### lock callback timer expired after 101s: evicting client at 192.168.0.100@o2ib ns: filter-lustre-OST004c_UUID lock: 0000000000bdf2e9/0x414d3785db3c23df lrc: 3/0,0 mode: PR/PR res: [0x1b0000040d:0x2:0x0].0x0 rrc: 2449 type: EXT [597700608->597934079] (req 597700608->597753855) gid 0 flags: 0x60000400030020 nid: 192.168.0.100@o2ib remote: 0xdfeed0111e7b6dfd expref: 5458 pid: 367974 timeout: 2494404 lvb_type: 0 [2494382.965612] LustreError: 364030:0:(ldlm_lockd.c:2517:ldlm_cancel_handler()) ldlm_cancel from 192.168.0.100@o2ib arrived at 1712126353 with bad export cookie 4705478233560533689 [2494382.966279] LustreError: 368013:0:(client.c:1255:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000c4cfe36c x1795273628412544/t0(0) o104->lustre-OST004c@192.168.0.100@o2ib:15/16 lens 328/224 e 0 to 0 dl 0 ref 1 fl Rpc:QU/0/ffffffff rc 0/-1 job:'' [2494382.983815] LustreError: 364030:0:(ldlm_lockd.c:2517:ldlm_cancel_handler()) Skipped 189 previous similar messages [2494383.009113] LustreError: 368013:0:(client.c:1255:ptlrpc_import_delay_req()) Skipped 482 previous similar messages [2494383.171856] Lustre: DEBUG MARKER: OSS: 2.15.4 [2494383.461960] LustreError: 328341:0:(ldlm_lockd.c:2517:ldlm_cancel_handler()) ldlm_cancel from 192.168.0.100@o2ib arrived at 1712126353 with bad export cookie 4705478233585896530 [2494383.480637] LustreError: 328341:0:(ldlm_lockd.c:2517:ldlm_cancel_handler()) Skipped 10676 previous similar messages [2494387.683179] LustreError: 364034:0:(ldlm_lockd.c:2517:ldlm_cancel_handler()) ldlm_cancel from 192.168.0.100@o2ib arrived at 1712126358 with bad export cookie 4705478233717782907 [2494387.701656] LustreError: 364034:0:(ldlm_lockd.c:2517:ldlm_cancel_handler()) Skipped 309 previous similar messages [2497866.246697] LustreError: 359945:0:(osd_io.c:76:__osd_init_iobuf()) ASSERTION( iobuf->dr_elapsed_valid == 0 ) failed: iobuf 00000000ba6abfbb, reqs 0, rw 1, line 1641 [2497866.264196] LustreError: 359945:0:(osd_io.c:76:__osd_init_iobuf()) LBUG [2497866.272901] Pid: 359945, comm: mdt_io01_004 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 SMP Wed Feb 21 13:52:43 CST 2024 [2497866.285484] Call Trace TBD: [2497866.289802] Kernel panic - not syncing: LBUG [2497866.295553] CPU: 25 PID: 359945 Comm: mdt_io01_004 Kdump: loaded Tainted: G OE 5.10.0-188.0.0.101.oe2203sp3.aarch64 #1 [2497866.309438] Hardware name: Huawei TaiShan 200 (Model 2280)/BC82AMDDA, BIOS 1.70 01/07/2021 [2497866.319682] Call trace: [2497866.323599] dump_backtrace+0x0/0x200 [2497866.328700] show_stack+0x20/0x30 [2497866.333451] dump_stack+0xf0/0x138 [2497866.338270] panic+0x194/0x39c [2497866.342745] lbug_with_loc+0x88/0x90 [libcfs] [2497866.348525] __osd_init_iobuf+0x2e8/0x3fc [osd_ldiskfs] [2497866.355155] osd_write_prep+0x8c/0x3b0 [osd_ldiskfs] [2497866.361546] mdt_preprw_write.constprop.0+0x68c/0x820 [mdt] [2497866.368504] mdt_obd_preprw+0x2f4/0x4d4 [mdt] [2497866.374386] tgt_brw_write+0x1414/0x2af0 [ptlrpc] [2497866.380506] tgt_handle_request0+0xd8/0x944 [ptlrpc] [2497866.386868] tgt_request_handle+0x2a0/0xda0 [ptlrpc] [2497866.393209] ptlrpc_server_handle_request.isra.0+0x3d4/0x1214 [ptlrpc] [2497866.401475] ptlrpc_main+0xdec/0x16d0 [ptlrpc] [2497866.407222] kthread+0x108/0x13c [2497866.411750] ret_from_fork+0x10/0x18 [2497866.416645] SMP: stopping secondary CPUs [2497866.424324] Starting crashdump kernel... [2497866.429519] Bye!