<ConMan> Console [trevis-106vm14] connected to <trevis-106:6013>.
[ 290.722917] kdump-tools[5938]: Starting kdump-tools:
[ 290.731653] kdump-tools[5945]: * Creating symlink /var/lib/kdump/vmlinuz
[ 290.773855] kdump-tools[5965]: kdump-tools: Generating /var/lib/kdump/initrd.img-5.15.0-126-generic
[ 308.127860] kdump-tools[5945]: * Creating symlink /var/lib/kdump/initrd.img
[ 309.092988] kdump-tools[5945]: * loaded kdump kernel
[ 367.333214] kdump-tools[24257]: Stopping kdump-tools:
[ 367.334183] kdump-tools[24264]: * unloaded kdump kernel
[ 367.382621] kdump-tools[24284]: Starting kdump-tools:
[ 367.383576] kdump-tools[24291]: * Creating symlink /var/lib/kdump/vmlinuz
[ 367.385934] kdump-tools[24291]: * Creating symlink /var/lib/kdump/initrd.img
[ 368.226419] kdump-tools[24291]: * loaded kdump kernel
<ConMan> Console [trevis-106vm14] disconnected from <trevis-106:6013>.
<ConMan> Console [trevis-106vm14] connected to <trevis-106:6013>.
..... ok
net0: 10.240.44.153/255.255.240.0 gw 10.240.32.1
Next server: 10.240.32.5
Filename: /pxelinux.0
tftp://10.240.32.5//pxelinux.0... ok
pxelinux.0 : 26759 bytes [PXE-NBP]
[1S
PXELINUX 4.05 0x5bd8f633 Copyright (C) 1994-2011 H. Peter Anvin et al[1S
!PXE entry point found (we hope) at 9C2C:0160 via plan A[1S
UNDI code segment at 9C2C len 0802[1S
UNDI data segment at 9CB0 len 2CE0[1S
Getting cached packet 01 02 03[1S
My IP address seems to be 0AF02C99 10.240.44.153[1S
ip=10.240.44.153:10.240.32.5:10.240.32.1:255.255.240.0[1S
BOOTIF=01-52-54-00-3e-d9-7f[1S
SYSUUID=10d44b74-1c64-444b-89e6-a7d1cc9fda86[1S
TFTP prefix: /[1S
Trying to load: pxelinux.cfg/10d44b74-1c64-444b-89e6-a7d1cc9fda86
Trying to load: pxelinux.cfg/01-52-54-00-3e-d9-7f ok[1S
Booting from local disk...[1S
No more network devices
Press Ctrl-B for the iPXE command line...
Booting from Hard Disk...
Loading Linux 5.15.0-94-generic ...
Loading initial ramdisk ...
[ 0.000000] Linux version 5.15.0-94-generic (buildd@lcy02-amd64-096) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 (Ubuntu 5.15.0-94.104-generic 5.15.136)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=a72e3805-c7e8-4bb3-840b-92bd865f2f09 ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 crashkernel=512M-:296M
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Hygon HygonGenuine
[ 0.000000] Centaur CentaurHauls
[ 0.000000] zhaoxin Shanghai
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfff9fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000bfffa000-0x00000000bfffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.4 present.
[ 0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr b3201001, primary cpu clock
[ 0.000005] kvm-clock: using sched offset of 22680378644 cycles
[ 0.000012] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000024] tsc: Detected 2294.686 MHz processor
[ 0.000299] last_pfn = 0xbfffa max_arch_pfn = 0x400000000
[ 0.000349] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.012212] found SMP MP-table at [mem 0x000f6390-0x000f639f]
[ 0.012255] Using GB pages for direct mapping
[ 0.012421] RAMDISK: [mem 0x2b8bf000-0x31c56fff]
[ 0.012456] ACPI: Early table checksum verification disabled
[ 0.012486] ACPI: RSDP 0x00000000000F61C0 000014 (v00 BOCHS )
[ 0.012498] ACPI: RSDT 0x00000000BFFFFB40 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.012517] ACPI: FACP 0x00000000BFFFF177 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.012534] ACPI: DSDT 0x00000000BFFFE040 001137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.012543] ACPI: FACS 0x00000000BFFFE000 000040
[ 0.012548] ACPI: SSDT 0x00000000BFFFF1EB 0008D5 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.012554] ACPI: APIC 0x00000000BFFFFAC0 000080 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.012557] ACPI: Reserving FACP table memory at [mem 0xbffff177-0xbffff1ea]
[ 0.012560] ACPI: Reserving DSDT table memory at [mem 0xbfffe040-0xbffff176]
[ 0.012561] ACPI: Reserving FACS table memory at [mem 0xbfffe000-0xbfffe03f]
[ 0.012562] ACPI: Reserving SSDT table memory at [mem 0xbffff1eb-0xbffffabf]
[ 0.012564] ACPI: Reserving APIC table memory at [mem 0xbffffac0-0xbffffb3f]
[ 0.013081] No NUMA configuration found
[ 0.013083] Faking a node at [mem 0x0000000000000000-0x00000000bfff9fff]
[ 0.013102] NODE_DATA(0) allocated [mem 0xbffd0000-0xbfff9fff]
[ 0.013621] Reserving 296MB of memory at 2512MB for crashkernel (System RAM: 3071MB)
[ 0.013666] Zone ranges:
[ 0.013674] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.013680] DMA32 [mem 0x0000000001000000-0x00000000bfff9fff]
[ 0.013686] Normal empty
[ 0.013688] Device empty
[ 0.013689] Movable zone start for each node
[ 0.013695] Early memory node ranges
[ 0.013696] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.013698] node 0: [mem 0x0000000000100000-0x00000000bfff9fff]
[ 0.013700] Initmem setup node 0 [mem 0x0000000000001000-0x00000000bfff9fff]
[ 0.013728] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.014002] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.056800] On node 0, zone DMA32: 6 pages in unavailable ranges
[ 0.057352] ACPI: PM-Timer IO Port: 0x608
[ 0.057383] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.057437] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.057443] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.057446] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.057453] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.057458] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.057460] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.057465] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.057472] TSC deadline timer available
[ 0.057474] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.057512] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.057515] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.057517] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[ 0.057518] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[ 0.057523] [mem 0xc0000000-0xfeffbfff] available for PCI devices
[ 0.057525] Booting paravirtualized kernel on KVM
[ 0.057535] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.057551] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.059880] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u1048576
[ 0.059919] kvm-guest: stealtime: cpu 0, msr bcc33080
[ 0.059923] kvm-guest: PV spinlocks enabled
[ 0.059926] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)
[ 0.059944] Built 1 zonelists, mobility grouping on. Total pages: 773882
[ 0.059946] Policy zone: DMA32
[ 0.059948] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=a72e3805-c7e8-4bb3-840b-92bd865f2f09 ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 crashkernel=512M-:296M
[ 0.060284] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic biosdevname=0", will be passed to user space.
[ 0.064631] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.066817] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.066905] mem auto-init: stack:off, heap alloc:on, heap free:off
[ 0.075132] Memory: 2625900K/3145312K available (16393K kernel code, 4394K rwdata, 10876K rodata, 3356K init, 18716K bss, 519152K reserved, 0K cma-reserved)
[ 0.075668] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.075708] Kernel/User page tables isolation: enabled
[ 0.075808] ftrace: allocating 50666 entries in 198 pages
[ 0.106371] ftrace: allocated 198 pages with 4 groups
[ 0.106740] rcu: Hierarchical RCU implementation.
[ 0.106743] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=2.
[ 0.106745] Rude variant of Tasks RCU enabled.
[ 0.106746] Tracing variant of Tasks RCU enabled.
[ 0.106750] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.106751] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.112163] NR_IRQS: 524544, nr_irqs: 440, preallocated irqs: 16
[ 0.112367] random: crng init done
[ 0.113321] Console: colour VGA+ 80x25
[ 0.123901] printk: console [tty0] enabled
[ 0.225784] printk: console [ttyS0] enabled
[ 0.226566] ACPI: Core revision 20210730
[ 0.227393] APIC: Switch to symmetric I/O mode setup
[ 0.228593] x2apic enabled
[ 0.229477] Switched APIC routing to physical x2apic.
[ 0.231826] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.233052] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x211399552f8, max_idle_ns: 440795292447 ns
[ 0.235015] Calibrating delay loop (skipped) preset value.. 4589.37 BogoMIPS (lpj=9178744)
[ 0.239131] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.240170] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.241307] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.243014] Spectre V2 : Mitigation: Retpolines
[ 0.243904] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.245439] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.247012] Speculative Store Bypass: Vulnerable
[ 0.247919] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[ 0.249137] TAA: Vulnerable: Clear CPU buffers attempted, no microcode
[ 0.251012] MMIO Stale Data: Unknown: No mitigations
[ 0.251965] SRBDS: Unknown: Dependent on hypervisor status
[ 0.253031] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.254489] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.255012] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.256202] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.257359] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.280378] Freeing SMP alternatives memory: 44K
[ 0.281324] pid_max: default: 32768 minimum: 301
[ 0.282252] LSM: Security Framework initializing
[ 0.283031] landlock: Up and running.
[ 0.283775] Yama: becoming mindful.
[ 0.284554] AppArmor: AppArmor initialized
[ 0.285472] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.287072] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.289113] smpboot: CPU0: Intel Core Processor (Broadwell) (family: 0x6, model: 0x3d, stepping: 0x2)
[ 0.291010] Performance Events: unsupported p6 CPU model 61 no PMU driver, software events only.
[ 0.291010] signal: max sigframe size: 1776
[ 0.291056] rcu: Hierarchical SRCU implementation.
[ 0.292674] NMI watchdog: Perf NMI watchdog permanently disabled
[ 0.293945] smp: Bringing up secondary CPUs ...
[ 0.295106] x86: Booting SMP configuration:
[ 0.295956] .... node #0, CPUs: #1
[ 0.122651] kvm-clock: cpu 1, msr b3201041, secondary cpu clock
[ 0.122651] smpboot: CPU 1 Converting physical 0 to logical die 1
[ 0.299034] kvm-guest: stealtime: cpu 1, msr bcd33080
[ 0.299983] smp: Brought up 1 node, 2 CPUs
[ 0.299983] smpboot: Max logical packages: 2
[ 0.300523] smpboot: Total of 2 processors activated (9178.74 BogoMIPS)
[ 0.303130] devtmpfs: initialized
[ 0.303719] x86/mm: Memory block size: 128MB
[ 0.304957] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.304957] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.307090] pinctrl core: initialized pinctrl subsystem
[ 0.308258] PM: RTC time: 12:35:19, date: 2024-12-16
[ 0.309374] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.310972] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[ 0.311477] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[ 0.313225] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[ 0.314543] audit: initializing netlink subsys (disabled)
[ 0.315066] audit: type=2000 audit(1734352521.572:1): state=initialized audit_enabled=0 res=1
[ 0.315233] thermal_sys: Registered thermal governor 'fair_share'
[ 0.317028] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.318084] thermal_sys: Registered thermal governor 'step_wise'
[ 0.319020] thermal_sys: Registered thermal governor 'user_space'
[ 0.320124] thermal_sys: Registered thermal governor 'power_allocator'
[ 0.321172] EISA bus registered
[ 0.322905] cpuidle: using governor ladder
[ 0.323018] cpuidle: using governor menu
[ 0.323861] ACPI: bus type PCI registered
[ 0.324588] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 0.325927] PCI: Using configuration type 1 for base access
[ 0.328505] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.330036] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.331019] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.335055] ACPI: Added _OSI(Module Device)
[ 0.335851] ACPI: Added _OSI(Processor Device)
[ 0.336635] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.337459] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.338383] ACPI: Added _OSI(Linux-Dell-Video)
[ 0.339015] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[ 0.339923] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[ 0.342337] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.343882] ACPI: Interpreter enabled
[ 0.344583] ACPI: PM: (supports S0 S5)
[ 0.345256] ACPI: Using IOAPIC for interrupt routing
[ 0.346132] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.347022] PCI: Using E820 reservations for host bridge windows
[ 0.348307] ACPI: Enabled 16 GPEs in block 00 to 0F
[ 0.352316] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.353407] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[ 0.354744] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 0.355405] acpiphp: Slot [3] registered
[ 0.356128] acpiphp: Slot [4] registered
[ 0.356854] acpiphp: Slot [5] registered
[ 0.357578] acpiphp: Slot [6] registered
[ 0.358295] acpiphp: Slot [7] registered
[ 0.359031] acpiphp: Slot [8] registered
[ 0.359760] acpiphp: Slot [9] registered
[ 0.360493] acpiphp: Slot [10] registered
[ 0.361227] acpiphp: Slot [11] registered
[ 0.361981] acpiphp: Slot [12] registered
[ 0.362729] acpiphp: Slot [13] registered
[ 0.363042] acpiphp: Slot [14] registered
[ 0.363777] acpiphp: Slot [15] registered
[ 0.364510] acpiphp: Slot [16] registered
[ 0.365256] acpiphp: Slot [17] registered
[ 0.365999] acpiphp: Slot [18] registered
[ 0.366747] acpiphp: Slot [19] registered
[ 0.367042] acpiphp: Slot [20] registered
[ 0.367780] acpiphp: Slot [21] registered
[ 0.368541] acpiphp: Slot [22] registered
[ 0.369273] acpiphp: Slot [23] registered
[ 0.370012] acpiphp: Slot [24] registered
[ 0.370747] acpiphp: Slot [25] registered
[ 0.371041] acpiphp: Slot [26] registered
[ 0.371783] acpiphp: Slot [27] registered
[ 0.372522] acpiphp: Slot [28] registered
[ 0.373272] acpiphp: Slot [29] registered
[ 0.374004] acpiphp: Slot [30] registered
[ 0.374740] acpiphp: Slot [31] registered
[ 0.375030] PCI host bridge to bus 0000:00
[ 0.375763] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.376695] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.377856] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.378974] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 0.379014] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[ 0.380364] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[ 0.382238] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[ 0.383814] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[ 0.390358] pci 0000:00:01.1: reg 0x20: [io 0xc0a0-0xc0af]
[ 0.393291] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.394494] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.395015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.396198] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.397557] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
[ 0.403588] pci 0000:00:01.2: reg 0x20: [io 0xc040-0xc05f]
[ 0.407162] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[ 0.408784] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.410017] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.411827] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
[ 0.415014] pci 0000:00:03.0: reg 0x10: [io 0xc060-0xc07f]
[ 0.418135] pci 0000:00:03.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff]
[ 0.429142] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
[ 0.431137] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000
[ 0.434418] pci 0000:00:04.0: reg 0x10: [io 0xc000-0xc03f]
[ 0.437201] pci 0000:00:04.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff]
[ 0.448916] pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00
[ 0.451015] pci 0000:00:05.0: reg 0x10: [io 0xc080-0xc09f]
[ 0.464201] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[ 0.465382] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[ 0.466534] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
[ 0.467137] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[ 0.468227] ACPI: PCI: Interrupt link LNKS configured for IRQ 9
[ 0.469583] iommu: Default domain type: Translated
[ 0.469583] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.471274] SCSI subsystem initialized
[ 0.472019] vgaarb: loaded
[ 0.472019] ACPI: bus type USB registered
[ 0.472352] usbcore: registered new interface driver usbfs
[ 0.473305] usbcore: registered new interface driver hub
[ 0.475022] usbcore: registered new device driver usb
[ 0.475929] pps_core: LinuxPPS API ver. 1 registered
[ 0.476789] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.478265] PTP clock support registered
[ 0.479029] EDAC MC: Ver: 3.0.0
[ 0.479876] NetLabel: Initializing
[ 0.479876] NetLabel: domain hash size = 128
[ 0.480628] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO
[ 0.481616] NetLabel: unlabeled traffic allowed by default
[ 0.483042] PCI: Using ACPI for IRQ routing
[ 0.485957] clocksource: Switched to clocksource kvm-clock
[ 0.509316] VFS: Disk quotas dquot_6.6.0
[ 0.510105] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.511478] AppArmor: AppArmor Filesystem Enabled
[ 0.512330] pnp: PnP ACPI init
[ 0.513435] pnp: PnP ACPI: found 5 devices
[ 0.521085] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.522630] NET: Registered PF_INET protocol family
[ 0.523954] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.527935] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.529606] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.531138] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.532924] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[ 0.534179] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.535553] MPTCP token hash table entries: 4096 (order: 4, 98304 bytes, linear)
[ 0.536890] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.538088] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.539337] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.540314] NET: Registered PF_XDP protocol family
[ 0.541172] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.542221] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.543270] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[ 0.544428] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
[ 0.545628] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.546627] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.547652] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.549022] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[ 0.550596] PCI: CLS 0 bytes, default 64
[ 0.551428] Trying to unpack rootfs image as initramfs...
[ 0.558761] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x211399552f8, max_idle_ns: 440795292447 ns
[ 0.567359] Initialise system trusted keyrings
[ 0.568201] Key type blacklist registered
[ 0.574686] workingset: timestamp_bits=36 max_order=20 bucket_order=0
[ 0.577453] zbud: loaded
[ 0.578325] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.579553] fuse: init (API version 7.34)
[ 0.580513] integrity: Platform Keyring initialized
[ 0.591279] Key type asymmetric registered
[ 0.592047] Asymmetric key parser 'x509' registered
[ 0.592918] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
[ 0.594224] io scheduler mq-deadline registered
[ 0.595356] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 0.596628] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.597913] ACPI: button: Power Button [PWRF]
[ 0.599370] ACPI: \_SB_.LNKC: Enabled at IRQ 10
[ 0.600244] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
[ 0.601721] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[ 0.603775] ACPI: \_SB_.LNKA: Enabled at IRQ 10
[ 0.604629] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[ 0.606804] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[ 0.630638] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 0.635358] Linux agpgart interface v0.103
[ 0.639822] loop: module loaded
[ 0.646692] scsi host0: ata_piix
[ 0.647507] scsi host1: ata_piix
[ 0.648168] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
[ 0.649315] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
[ 0.650945] tun: Universal TUN/TAP device driver, 1.6
[ 0.651882] PPP generic driver version 2.4.2
[ 0.652988] VFIO - User Level meta-driver version: 0.3
[ 0.653974] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.655093] ehci-pci: EHCI PCI platform driver
[ 0.655893] ehci-platform: EHCI generic platform driver
[ 0.656806] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.657851] ohci-pci: OHCI PCI platform driver
[ 0.658644] ohci-platform: OHCI generic platform driver
[ 0.659562] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.661747] uhci_hcd 0000:00:01.2: UHCI Host Controller
[ 0.662682] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
[ 0.663936] uhci_hcd 0000:00:01.2: detected 2 ports
[ 0.664912] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
[ 0.665987] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 5.15
[ 0.667405] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 0.668636] usb usb1: Product: UHCI Host Controller
[ 0.669495] usb usb1: Manufacturer: Linux 5.15.0-94-generic uhci_hcd
[ 0.670589] usb usb1: SerialNumber: 0000:00:01.2
[ 0.671554] hub 1-0:1.0: USB hub found
[ 0.672275] hub 1-0:1.0: 2 ports detected
[ 0.673226] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[ 0.675521] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.676420] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.677447] mousedev: PS/2 mouse device common for all mice
[ 0.678545] rtc_cmos 00:00: RTC can wake from S4
[ 0.680195] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[ 0.681918] rtc_cmos 00:00: registered as rtc0
[ 0.682878] rtc_cmos 00:00: setting system clock to 2024-12-16T12:35:19 UTC (1734352519)
[ 0.684383] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
[ 0.685455] i2c_dev: i2c /dev entries driver
[ 0.686283] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[ 0.688346] device-mapper: uevent: version 1.0.3
[ 0.689241] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[ 0.690693] platform eisa.0: Probing EISA bus 0
[ 0.691519] platform eisa.0: EISA: Cannot allocate resource for mainboard
[ 0.692668] platform eisa.0: Cannot allocate resource for EISA slot 1
[ 0.693767] platform eisa.0: Cannot allocate resource for EISA slot 2
[ 0.694865] platform eisa.0: Cannot allocate resource for EISA slot 3
[ 0.695968] platform eisa.0: Cannot allocate resource for EISA slot 4
[ 0.697072] platform eisa.0: Cannot allocate resource for EISA slot 5
[ 0.698167] platform eisa.0: Cannot allocate resource for EISA slot 6
[ 0.699277] platform eisa.0: Cannot allocate resource for EISA slot 7
[ 0.700391] platform eisa.0: Cannot allocate resource for EISA slot 8
[ 0.701486] platform eisa.0: EISA: Detected 0 cards
[ 0.702329] intel_pstate: CPU model not supported
[ 0.703211] ledtrig-cpu: registered to indicate activity on CPUs
[ 0.704335] drop_monitor: Initializing network drop monitor service
[ 0.705628] NET: Registered PF_INET6 protocol family
[ 1.600203] Freeing initrd memory: 101984K
[ 1.607314] Segment Routing with IPv6
[ 1.608061] In-situ OAM (IOAM) with IPv6
[ 1.608814] NET: Registered PF_PACKET protocol family
[ 1.609768] Key type dns_resolver registered
[ 1.610896] IPI shorthand broadcast: enabled
[ 1.611712] sched_clock: Marking stable (1492131213, 118651859)->(1650465589, -39682517)
[ 1.613259] registered taskstats version 1
[ 1.614084] Loading compiled-in X.509 certificates
[ 1.615717] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[ 1.617930] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969'
[ 1.620148] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19'
[ 1.621803] blacklist: Loading compiled-in revocation X.509 certificates
[ 1.622954] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing: 61482aa2830d0ab2ad5af10b7250da9033ddcef0'
[ 1.624609] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2017): 242ade75ac4a15e50d50c84b0d45ff3eae707a03'
[ 1.626295] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (ESM 2018): 365188c1d374d6b07c3c8f240f8ef722433d6a8b'
[ 1.628039] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2019): c0746fd6c5da3ae827864651ad66ae47fe24b3e8'
[ 1.629741] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d'
[ 1.631501] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v2): 4cf046892d6fd3c9a5b03f98d845f90851dc6a8c'
[ 1.633226] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v3): 100437bb6de6e469b581e61cd66bce3ef4ed53af'
[ 1.634958] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9'
[ 1.636989] zswap: loaded using pool lzo/zbud
[ 1.638023] Key type .fscrypt registered
[ 1.638764] Key type fscrypt-provisioning registered
[ 1.642299] Key type encrypted registered
[ 1.643051] AppArmor: AppArmor sha1 policy hashing enabled
[ 1.644001] ima: No TPM chip found, activating TPM-bypass!
[ 1.644943] Loading compiled-in module X.509 certificates
[ 1.646454] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[ 1.648080] ima: Allocated hash algorithm: sha1
[ 1.648881] ima: No architecture policies found
[ 1.649683] evm: Initialising EVM extended attributes:
[ 1.650563] evm: security.selinux
[ 1.651193] evm: security.SMACK64
[ 1.651800] evm: security.SMACK64EXEC
[ 1.652466] evm: security.SMACK64TRANSMUTE
[ 1.653200] evm: security.SMACK64MMAP
[ 1.653855] evm: security.apparmor
[ 1.654480] evm: security.ima
[ 1.655041] evm: security.capability
[ 1.655694] evm: HMAC attrs: 0x1
[ 1.656626] PM: Magic number: 4:541:582
[ 1.657614] RAS: Correctable Errors collector initialized.
[ 1.660421] Freeing unused decrypted memory: 2036K
[ 1.662709] Freeing unused kernel image (initmem) memory: 3356K
[ 1.678675] Write protecting the kernel read-only data: 30720k
[ 1.681728] Freeing unused kernel image (text/rodata gap) memory: 2036K
[ 1.684198] Freeing unused kernel image (rodata/data gap) memory: 1412K
[ 1.743237] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 1.744375] x86/mm: Checking user space page tables
[ 1.799099] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 1.800214] Run /init as init process
Loading, please wait...
Starting version 249.11-0ubuntu3.12
[ 1.936179] virtio_blk virtio1: [vda] 209715200 512-byte logical blocks (107 GB/100 GiB)
[ 1.938545] cryptd: max_cpu_qlen set to 1000
[ 1.948269] AVX2 version of gcm_enc/dec engaged.
[ 1.949230] AES CTR mode by8 optimization enabled
[ 1.954769] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
[ 1.957921] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
[ 1.959308] vda: vda1
[ 1.959704] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
[ 1.971294] FDC 0 is a S82078B
Begin: Loading essential drivers ... [ 3.214670] raid6: avx2x4 gen() 20027 MB/s
[ 3.282662] raid6: avx2x4 xor() 9728 MB/s
[ 3.350662] raid6: avx2x2 gen() 25602 MB/s
[ 3.418661] raid6: avx2x2 xor() 15992 MB/s
[ 3.486662] raid6: avx2x1 gen() 23833 MB/s
[ 3.554661] raid6: avx2x1 xor() 13963 MB/s
[ 3.622662] raid6: sse2x4 gen() 14558 MB/s
[ 3.690662] raid6: sse2x4 xor() 8181 MB/s
[ 3.758662] raid6: sse2x2 gen() 13875 MB/s
[ 3.826661] raid6: sse2x2 xor() 9167 MB/s
[ 3.894662] raid6: sse2x1 gen() 12223 MB/s
[ 3.966663] raid6: sse2x1 xor() 8210 MB/s
[ 3.967458] raid6: using algorithm avx2x2 gen() 25602 MB/s
[ 3.968390] raid6: .... xor() 15992 MB/s, rmw enabled
[ 3.969263] raid6: using avx2x2 recovery algorithm
[ 3.971669] xor: automatically using best checksumming function avx
[ 3.973801] async_tx: api initialized (async)
done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... [ 4.046082] Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes
Scanning for Btrfs filesystems
done.
Begin: Will now check root file system ... fsck from util-linux 2.37.2
[/usr/sbin/fsck.ext4 (1) -- /dev/vda1] fsck.ext4 -a -C0 /dev/vda1
/dev/vda1: recovering journal
/dev/vda1: clean, 146996/1310720 files, 2379029/5242880 blocks
done.
[ 5.172295] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[ 5.336560] systemd[1]: Inserted module 'autofs4'
[ 5.346903] systemd[1]: systemd 249.11-0ubuntu3.12 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 5.351938] systemd[1]: Detected virtualization kvm.
[ 5.352832] systemd[1]: Detected architecture x86-64.
Welcome to [1mUbuntu 22.04 LTS[0m!
[ 5.354919] systemd[1]: Hostname set to <trevis-106vm14.trevis.whamcloud.com>.
[ 5.438605] systemd[1]: Configuration file /run/systemd/system/netplan-ovs-cleanup.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
[ 5.493964] systemd[1]: Queued start job for default target Graphical Interface.
[ 5.496445] systemd[1]: Created slice Slice /system/modprobe.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/modprobe[0m.
[ 5.498567] systemd[1]: Created slice Slice /system/postfix.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/postfix[0m.
[ 5.500589] systemd[1]: Created slice Slice /system/serial-getty.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/serial-getty[0m.
[ 5.502569] systemd[1]: Created slice User and Session Slice.
[[0;32m OK [0m] Created slice [0;1;39mUser and Session Slice[0m.
[ 5.504350] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[[0;32m OK [0m] Started [0;1;39mForward Password R…uests to Wall Directory Watch[0m.
[ 5.506690] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[[0;32m OK [0m] Set up automount [0;1;39mArbitrary…s File System Automount Point[0m.
[ 5.509183] systemd[1]: Reached target Slice Units.
[[0;32m OK [0m] Reached target [0;1;39mSlice Units[0m.
[ 5.510685] systemd[1]: Reached target System Time Set.
[[0;32m OK [0m] Reached target [0;1;39mSystem Time Set[0m.
[ 5.512264] systemd[1]: Reached target Local Verity Protected Volumes.
[[0;32m OK [0m] Reached target [0;1;39mLocal Verity Protected Volumes[0m.
[ 5.514199] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[[0;32m OK [0m] Listening on [0;1;39mDevice-mapper event daemon FIFOs[0m.
[ 5.516242] systemd[1]: Listening on LVM2 poll daemon socket.
[[0;32m OK [0m] Listening on [0;1;39mLVM2 poll daemon socket[0m.
[ 5.518033] systemd[1]: Listening on multipathd control socket.
[[0;32m OK [0m] Listening on [0;1;39mmultipathd control socket[0m.
[ 5.523333] systemd[1]: Listening on RPCbind Server Activation Socket.
[[0;32m OK [0m] Listening on [0;1;39mRPCbind Server Activation Socket[0m.
[ 5.525604] systemd[1]: Listening on Syslog Socket.
[[0;32m OK [0m] Listening on [0;1;39mSyslog Socket[0m.
[ 5.527210] systemd[1]: Listening on fsck to fsckd communication Socket.
[[0;32m OK [0m] Listening on [0;1;39mfsck to fsckd communication Socket[0m.
[ 5.529207] systemd[1]: Listening on initctl Compatibility Named Pipe.
[[0;32m OK [0m] Listening on [0;1;39minitctl Compatibility Named Pipe[0m.
[ 5.531299] systemd[1]: Listening on Journal Audit Socket.
[[0;32m OK [0m] Listening on [0;1;39mJournal Audit Socket[0m.
[ 5.533053] systemd[1]: Listening on Journal Socket (/dev/log).
[[0;32m OK [0m] Listening on [0;1;39mJournal Socket (/dev/log)[0m.
[ 5.534908] systemd[1]: Listening on Journal Socket.
[[0;32m OK [0m] Listening on [0;1;39mJournal Socket[0m.
[ 5.536566] systemd[1]: Listening on Network Service Netlink Socket.
[[0;32m OK [0m] Listening on [0;1;39mNetwork Service Netlink Socket[0m.
[ 5.538535] systemd[1]: Listening on udev Control Socket.
[[0;32m OK [0m] Listening on [0;1;39mudev Control Socket[0m.
[ 5.540258] systemd[1]: Listening on udev Kernel Socket.
[[0;32m OK [0m] Listening on [0;1;39mudev Kernel Socket[0m.
[ 5.542733] systemd[1]: Mounting Huge Pages File System...
Mounting [0;1;39mHuge Pages File System[0m...
[ 5.545183] systemd[1]: Mounting POSIX Message Queue File System...
Mounting [0;1;39mPOSIX Message Queue File System[0m...
[ 5.547637] systemd[1]: Mounting NFSD configuration filesystem...
Mounting [0;1;39mNFSD configuration filesystem[0m...
[ 5.550268] systemd[1]: Mounting Kernel Debug File System...
Mounting [0;1;39mKernel Debug File System[0m...
[ 5.552732] systemd[1]: Mounting Kernel Trace File System...
Mounting [0;1;39mKernel Trace File System[0m...
[ 5.556111] systemd[1]: Starting Journal Service...
Starting [0;1;39mJournal Service[0m...
[ 5.557813] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS being skipped.
[ 5.560496] systemd[1]: Starting Set the console keyboard layout...
Starting [0;1;39mSet the console keyboard layout[0m...
[ 5.573950] systemd[1]: Starting Create List of Static Device Nodes...
Starting [0;1;39mCreate List of Static Device Nodes[0m...
[ 5.580869] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
Starting [0;1;39mMonitoring of LVM…meventd or progress polling[0m...
[ 5.589408] systemd[1]: Condition check resulted in LXD - agent being skipped.
[ 5.592063] systemd[1]: Starting Load Kernel Module configfs...
Starting [0;1;39mLoad Kernel Module configfs[0m...
[ 5.595982] systemd[1]: Starting Load Kernel Module drm...
[ 5.599623] RPC: Registered named UNIX socket transport module.
Starting [0;1;39mLoad Kernel Module drm[0m...[ 5.601183] RPC: Registered udp transport module.
[ 5.602066] RPC: Registered tcp transport module.
[ 5.602949] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 5.618836] systemd[1]: Starting Load Kernel Module efi_pstore...
Starting [0;1;39mLoad Kernel Module efi_pstore[0m...
[ 5.626217] systemd[1]: Starting Load Kernel Module fuse...
Starting [0;1;39mLoad Kernel Module fuse[0m...
[ 5.629918] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
[ 5.631797] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[ 5.634462] systemd[1]: Starting Load Kernel Modules...
Starting [0;1;39mLoad Kernel Modules[0m...
[ 5.641807] systemd[1]: Starting Remount Root and Kernel File Systems...
Starting [0;1;39mRemount Root and Kernel File Systems[0m...
[ 5.648254] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[ 5.651838] IPMI message handler: version 39.2
[ 5.654573] ipmi device interface
[ 5.656688] systemd[1]: Starting Coldplug All udev Devices...
Starting [0;1;39mColdplug All udev Devices[0m...
[ 5.661302] systemd[1]: Started Journal Service.
[[0;32m OK [0m] Started [0;1;39mJournal Service[0m.
[ 5.670970] EXT4-fs (vda1): re-mounted. Opts: (null). Quota mode: none.
[[0;32m OK [0m] Mounted [0;1;39mHuge Pages File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mPOSIX Message Queue File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mNFSD configuration filesystem[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Debug File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Trace File System[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate List of Static Device Nodes[0m.
[[0;32m OK [0m] Finished [0;1;39mMonitoring of LVM… dmeventd or progress polling[0m.
[[0;32m OK [0m] Finished [0;1;39mSet the console keyboard layout[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module configfs[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module drm[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module efi_pstore[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module fuse[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Modules[0m.
[[0;32m OK [0m] Finished [0;1;39mRemount Root and Kernel File Systems[0m.
Activating swap [0;1;39m/swap.img[0m...
Mounting [0;1;39mFUSE Control File System[0m...
Mounting [0;1;39mKernel Configuration File System[0m...
Starting [0;1;39mDevice-Mapper Multipath Device Controller[0m...
Starting [0;1;39mFlush Journal to Persistent Storage[0m...
Starting [0;1;39mLoad/Save Random Seed[0m...
Starting [0;1;39mApply Kernel Variables[0m...
Starting [0;1;39mCreate System Users[0m...
[[0;32m OK [0m] Mounted [0;1;39mFUSE Control File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Configuration File System[0m.
[[0;32m OK [0m] Finished [0;1;39mColdplug All udev Devices[0m.
[ 5.821757] Adding 4194300k swap on /swap.img. Priority:-2 extents:4 across:5857276k FS
Starting [0;1;39mHelper to synchronize boot up for ifupdown[0m...
[ 5.827808] alua: device handler registered
[ 5.829711] emc: device handler registered
[ 5.832442] rdac: device handler registered
[ 5.832953] systemd-journald[354]: Received client request to flush runtime journal.
[[0;32m OK [0m] Activated swap [0;1;39m/swap.img[0m.
[[0;32m OK [0m] Reached target [0;1;39mSwaps[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate System Users[0m.
Starting [0;1;39mCreate Static Device Nodes in /dev[0m...
[ 5.868267] systemd-journald[354]: File /var/log/journal/2893385f3d7c4e94bd25f791e10d4326/system.journal corrupted or uncleanly shut down, renaming and replacing.
[[0;32m OK [0m] Finished [0;1;39mApply Kernel Variables[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad/Save Random Seed[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate Static Device Nodes in /dev[0m.
Starting [0;1;39mRule-based Manage…for Device Events and Files[0m...
[[0;32m OK [0m] Finished [0;1;39mFlush Journal to Persistent Storage[0m.
[[0;32m OK [0m] Started [0;1;39mDevice-Mapper Multipath Device Controller[0m.
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Local File Systems[0m.
[[0;32m OK [0m] Reached target [0;1;39mLocal File Systems[0m.
Starting [0;1;39mLoad AppArmor profiles[0m...
Starting [0;1;39mSet console font and keymap[0m...
Starting [0;1;39mCreate final runt…dir for shutdown pivot root[0m...
Starting [0;1;39mTell Plymouth To Write Out Runtime Data[0m...
Starting [0;1;39mSet Up Additional Binary Formats[0m...
Starting [0;1;39mCreate Volatile Files and Directories[0m...
Starting [0;1;39mUncomplicated firewall[0m...
[[0;32m OK [0m] Started [0;1;39mRule-based Manager for Device Events and Files[0m.
[[0;32m OK [0m] Finished [0;1;39mSet console font and keymap[0m.
[ 6.006339] audit: type=1400 audit(1734352524.816:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=427 comm="apparmor_parser"
[ 6.008861] audit: type=1400 audit(1734352524.816:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=427 comm="apparmor_parser"
[[0;32m OK [0m] Finished [0;1;39mCreate final runt…e dir for shutdown pivot root[0m.
[[0;32m OK [0m] Finished [0;1;39mTell Plymouth To Write Out Runtime Data[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate Volatile Files and Directories[0m.
[[0;32m OK [0m] Finished [0;1;39mUncomplicated firewall[0m.
[ 6.017308] audit: type=1400 audit(1734352524.828:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lsb_release" pid=426 comm="apparmor_parser"
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Network[0m.
Mounting [0;1;39mArbitrary Executable File Formats File System[0m...
Mounting [0;1;39mRPC Pipe File System[0m...
[[0;32m OK [0m] Started [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[[0;32m OK [0m] Reached target [0;1;39mLocal Encrypted Volumes[0m.
Starting [0;1;39mRPC bind portmap service[0m...
Starting [0;1;39mNetwork Configuration[0m...
[ 6.051695] audit: type=1400 audit(1734352524.864:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=439 comm="apparmor_parser"
[ 6.054560] audit: type=1400 audit(1734352524.864:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=439 comm="apparmor_parser"
[ 6.057371] audit: type=1400 audit(1734352524.864:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=439 comm="apparmor_parser"
[ 6.057374] audit: type=1400 audit(1734352524.864:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/{,usr/}sbin/dhclient" pid=439 comm="apparmor_parser"
[ 6.057376] audit: type=1400 audit(1734352524.868:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=440 comm="apparmor_parser"
[ 6.057379] audit: type=1400 audit(1734352524.868:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=440 comm="apparmor_parser"
[ 6.057381] audit: type=1400 audit(1734352524.868:11): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_groff" pid=440 comm="apparmor_parser"
Starting [0;1;39mRecord System Boot/Shutdown in UTMP[0m...
[[0;32m OK [0m] Started [0;1;39mRPC bind portmap service[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad AppArmor profiles[0m.
[[0;32m OK [0m] Mounted [0;1;39mArbitrary Executable File Formats File System[0m.
[[0;32m OK [0m] Finished [0;1;39mSet Up Additional Binary Formats[0m.
[[0;32m OK [0m] Mounted [0;1;39mRPC Pipe File System[0m.
[[0;32m OK [0m] Finished [0;1;39mRecord System Boot/Shutdown in UTMP[0m.
[[0;32m OK [0m] Reached target [0;1;39mrpc_pipefs.target[0m.
[[0;32m OK [0m] Reached target [0;1;39mRPC Port Mapper[0m.
[[0;32m OK [0m] Reached target [0;1;39mSystem Initialization[0m.
[[0;32m OK [0m] Started [0;1;39mTrigger to poll fo…y enabled on GCP LTS non-pro)[0m.
[[0;32m OK [0m] Started [0;1;39mDaily Cleanup of Temporary Directories[0m.
[[0;32m OK [0m] Started [0;1;39mUbuntu Advantage Timer for running repeated jobs[0m.
[[0;32m OK [0m] Reached target [0;1;39mPath Units[0m.
[[0;32m OK [0m] Listening on [0;1;39mD-Bus System Message Bus Socket[0m.
[[0;32m OK [0m] Listening on [0;1;39mOpen-iSCSI iscsid Socket[0m.
[[0;32m OK [0m] Listening on [0;1;39mUUID daemon activation socket[0m.
[[0;32m OK [0m] Reached target [0;1;39mSocket Units[0m.
[[0;32m OK [0m] Reached target [0;1;39mBasic System[0m.
[[0;32m OK [0m] Started [0;1;39mD-Bus System Message Bus[0m.
Starting [0;1;39mNetwork Manager[0m...
[[0;32m OK [0m] Started [0;1;39mSave initial kernel messages after boot[0m.
Starting [0;1;39mRemove Stale Onli…t4 Metadata Check Snapshots[0m...
Starting [0;1;39mRecord successful boot for GRUB[0m...
[[0;32m OK [0m] Started [0;1;39mirqbalance daemon[0m.
Starting [0;1;39mKernel crash dump capture service[0m...
Starting [0;1;39mLSB: Execute the …-e command to reboot system[0m...
Starting [0;1;39mDispatcher daemon for systemd-networkd[0m...
Starting [0;1;39mpNFS block layout mapping daemon[0m...
Starting [0;1;39mNFSv4 ID-name mapping service[0m...
Starting [0;1;39mNFSv4 Client Tracking Daemon[0m...
Starting [0;1;39mAuthorization Manager[0m...
[[0;32m OK [0m] Reached target [0;1;39mNFS client services[0m.
Starting [0;1;39mSystem Logging Service[0m...
Starting [0;1;39mUser Login Management[0m...
Starting [0;1;39mDisk Manager[0m...
Starting [0;1;39mWPA supplicant[0m...
[[0;32m OK [0m] Started [0;1;39mNetwork Configuration[0m.
[[0;32m OK [0m] Finished [0;1;39mRemove Stale Onli…ext4 Metadata Check Snapshots[0m.
[[0;32m OK [0m] Started [0;1;39mpNFS block layout mapping daemon[0m.
[[0;32m OK [0m] Started [0;1;39mNFSv4 ID-name mapping service[0m.
[[0;32m OK [0m] Started [0;1;39mNFSv4 Client Tracking Daemon[0m.
[[0;32m OK [0m] Started [0;1;39mSystem Logging Service[0m.
[[0;32m OK [0m] Started [0;1;39mAuthorization Manager[0m.
[[0;32m OK [0m] Started [0;1;39mNetwork Manager[0m.
[[0;32m OK [0m] Started [0;1;39mWPA supplicant[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Execute the k…c -e command to reboot system[0m.
[[0;32m OK [0m] Finished [0;1;39mRecord successful boot for GRUB[0m.
[[0;32m OK [0m] Started [0;1;39mUser Login Management[0m.
Starting [0;1;39mModem Manager[0m...
Starting [0;1;39mNetwork Manager Wait Online[0m...
Starting [0;1;39mGRUB failed boot detection[0m...
Starting [0;1;39mHostname Service[0m...
Starting [0;1;39mWait for Network to be Configured[0m...
Starting [0;1;39mNetwork Name Resolution[0m...
[[0;32m OK [0m] Started [0;1;39mHostname Service[0m.
[[0;32m OK [0m] Started [0;1;39mDisk Manager[0m.
[[0;32m OK [0m] Found device [0;1;39m/dev/ttyS0[0m.
[ 6.532558] kdump-tools[467]: Starting kdump-tools:
[ 6.533576] kdump-tools[512]: * Creating symlink /var/lib/kdump/vmlinuz
Starting [0;1;39mNetwork Manager Script Dispatcher Service[0m...
[[0;32m OK [0m] Finished [0;1;39mGRUB failed boot detection[0m.
[[0;32m OK [0m] Started [0;1;39mNetwork Manager Script Dispatcher Service[0m.
[[0;32m OK [0m] Finished [0;1;39mNetwork Manager Wait Online[0m.
[[0;32m OK [0m] Started [0;1;39mNetwork Name Resolution[0m.
[[0;32m OK [0m] Reached target [0;1;39mHost and Network Name Lookups[0m.
[[0;32m OK [0m] Started [0;1;39mModem Manager[0m.
[ 6.710505] kdump-tools[613]: libkmod: ERROR ../libkmod/libkmod-module.c:1941 kmod_module_get_holders: could not open '/sys/module/joydev/holders': No such file or directory
[ 6.720375] kdump-tools[579]: kdump-tools: Generating /var/lib/kdump/initrd.img-5.15.0-94-generic
[[0;32m OK [0m] Started [0;1;39mDispatcher daemon for systemd-networkd[0m.
[[0;32m OK [0m] Listening on [0;1;39mLoad/Save RF …itch Status /dev/rfkill Watch[0m.
[[0;32m OK [0m] Found device [0;1;39mVirtio network device[0m.
[[0;32m OK [0m] Started [0;1;39mifup for eth0[0m.
[[0;32m OK [0m] Finished [0;1;39mHelper to synchronize boot up for ifupdown[0m.
Starting [0;1;39mRaise network interfaces[0m...
[ 10.091235] IPv6: eth0: IPv6 duplicate address fd33:3981:3213:f020:0:5254:82:c0a8 used by 52:54:00:6a:a7:e6 detected!
[ 21.760748] kdump-tools[512]: * Creating symlink /var/lib/kdump/initrd.img
[ 22.632816] kdump-tools[512]: * loaded kdump kernel
[[0;32m OK [0m] Finished [0;1;39mKernel crash dump capture service[0m.
[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (19s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (20s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (20s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (22s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (24s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (24s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (25s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (26s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (27s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (27s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (28s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (29s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (30s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (30s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (31s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (31s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (33s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (34s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (34s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (36s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (36s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (37s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (37s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (40s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (40s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (41s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (41s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (43s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (43s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (44s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (44s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (47s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (47s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (48s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (48s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (50s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (50s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (51s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (51s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (54s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (54s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (55s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (55s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (57s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (57s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (58s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (58s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…to be Configured (2min / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…to be Configured (2min / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (2min 1s / no limit)
M
[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mWait for Network to be Configured[0m.
[KSee 'systemctl status systemd-networkd-wait-online.service' for details.
[[0;32m OK [0m] Reached target [0;1;39mNetwork is Online[0m.
[[0;32m OK [0m] Started [0;1;39mDownload data for …ailed at package install time[0m.
[[0;32m OK [0m] Started [0;1;39mCheck to see wheth…w version of Ubuntu available[0m.
[[0;32m OK [0m] Reached target [0;1;39mTimer Units[0m.
Starting [0;1;39mNFS Mount Daemon[0m...
Starting [0;1;39mSamba NMB Daemon[0m...
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Remote File Systems[0m.
Mounting [0;1;39m/autotest[0m...
Mounting [0;1;39m/scratch[0m...
[[0;32m OK [0m] Finished [0;1;39mAvailability of block devices[0m.
Starting [0;1;39mPostfix Mail Transport Agent (instance -)[0m...
Starting [0;1;39mNFS status monitor for NFSv2/3 locking.[0m...
Starting [0;1;39mOpenBSD Secure Shell server[0m...
[[0;32m OK [0m] Started [0;1;39mNFS Mount Daemon[0m.
[ 126.831215] FS-Cache: Loaded
[[0;32m OK [0m] Started [0;1;39mNFS status monitor for NFSv2/3 locking.[0m.
Starting [0;1;39mNFS server and services[0m...
[[0;32m OK [0m] Started [0;1;39mSamba NMB Daemon[0m.
Starting [0;1;39mSamba SMB Daemon[0m...
[ 126.882739] FS-Cache: Netfs 'nfs' registered for caching
[[0;32m OK [0m] Started [0;1;39mOpenBSD Secure Shell server[0m.
[[0;32m OK [0m] Started [0;1;39mSamba SMB Daemon[0m.
[ 127.049023] NFS: Registering the id_resolver key type
[ 127.049982] Key type id_resolver registered
[ 127.050916] Key type id_legacy registered
[ 127.061807] NFSD: Using nfsdcld client tracking operations.
[ 127.062857] NFSD: no clients to reclaim, skipping NFSv4 grace period (net f0000000)
[[0;32m OK [0m] Mounted [0;1;39m/autotest[0m.
[[0;32m OK [0m] Finished [0;1;39mNFS server and services[0m.
Starting [0;1;39mNotify NFS peers of a restart[0m...
[[0;32m OK [0m] Started [0;1;39mNotify NFS peers of a restart[0m.
[[0;32m OK [0m] Mounted [0;1;39m/scratch[0m.
[[0;32m OK [0m] Reached target [0;1;39mRemote File Systems[0m.
Starting [0;1;39mLSB: automatic crash report generation[0m...
Starting [0;1;39mAutomounts filesystems on demand[0m...
[[0;32m OK [0m] Started [0;1;39mRegular background program processing daemon[0m.
Starting [0;1;39mLSB: Load kernel image with kexec[0m...
Starting [0;1;39mLSB: Start/Stop t…mote power manager) daemon.[0m...
Starting [0;1;39mPermit User Sessions[0m...
Starting [0;1;39mLSB: Start vmtouch daemon[0m...
Starting [0;1;39mLSB: Starts or stops the xinetd daemon.[0m...
[[0;32m OK [0m] Started [0;1;39mLSB: Load kernel image with kexec[0m.
[[0;32m OK [0m] Finished [0;1;39mPermit User Sessions[0m.
Starting [0;1;39mHold until boot process finishes up[0m...
Starting [0;1;39mTerminate Plymouth Boot Screen[0m...
[[0;32m OK [0m] Started [0;1;39mLSB: automatic crash report generation[0m.
[[0;32m OK [0m] Finished [0;1;39mHold until boot process finishes up[0m.
[[0;32m OK [0m] Finished [0;1;39mTerminate Plymouth Boot Screen[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Start vmtouch daemon[0m.
[[0;32m OK [0m] Started [0;1;39mSerial Getty on ttyS0[0m.
Starting [0;1;39mSet console scheme[0m...
[[0;32m OK [0m] Started [0;1;39mLSB: Start/Stop th…remote power manager) daemon.[0m.
[[0;32m OK [0m] Finished [0;1;39mSet console scheme[0m.
[[0;32m OK [0m] Started [0;1;39mAutomounts filesystems on demand[0m.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/getty[0m.
[[0;32m OK [0m] Started [0;1;39mGetty on tty1[0m.
[[0;32m OK [0m] Reached target [0;1;39mLogin Prompts[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Starts or stops the xinetd daemon.[0m.
[[0;32m OK [0m] Created slice [0;1;39mUser Slice of UID 0[0m.
Starting [0;1;39mUser Runtime Directory /run/user/0[0m...
[[0;32m OK [0m] Finished [0;1;39mUser Runtime Directory /run/user/0[0m.
Starting [0;1;39mUser Manager for UID 0[0m...
[[0;32m OK [0m] Started [0;1;39mPostfix Mail Transport Agent (instance -)[0m.
Starting [0;1;39mPostfix Mail Transport Agent[0m...
[[0;32m OK [0m] Finished [0;1;39mPostfix Mail Transport Agent[0m.
[[0;32m OK [0m] Reached target [0;1;39mMulti-User System[0m.
[[0;32m OK [0m] Reached target [0;1;39mGraphical Interface[0m.
Starting [0;1;39mRecord Runlevel Change in UTMP[0m...
[[0;32m OK [0m] Finished [0;1;39mRecord Runlevel Change in UTMP[0m.
[[0;32m OK [0m] Started [0;1;39mUser Manager for UID 0[0m.
[[0;32m OK [0m] Started [0;1;39mSession 3 of User root[0m.
Ubuntu 22.04 LTS trevis-106vm14.trevis.whamcloud.com ttyS0
trevis-106vm14 login: [ 3976.138729] Autotest: Nodes provisioned in 79 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3976.664557] Autotest: Starting node configuration (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
Stopping [0;1;39mSession 49 of User root[0m...
[[0;32m OK [0m] Removed slice [0;1;39mSlice /system/modprobe[0m.
[[0;32m OK [0m] Stopped target [0;1;39mGraphical Interface[0m.
[[0;32m OK [0m] Stopped target [0;1;39mMulti-User System[0m.
[[0;32m OK [0m] Stopped target [0;1;39mLogin Prompts[0m.
[[0;32m OK [0m] Stopped target [0;1;39mRPC Port Mapper[0m.
[[0;32m OK [0m] Stopped target [0;1;39mTimer Units[0m.
[[0;32m OK [0m] Stopped [0;1;39mDaily apt upgrade and clean activities[0m.
[[0;32m OK [0m] Stopped [0;1;39mDaily apt download activities[0m.
[[0;32m OK [0m] Stopped [0;1;39mDaily dpkg database backup timer[0m.
[[0;32m OK [0m] Stopped [0;1;39mPeriodic ext4 Onli…ata Check for All Filesystems[0m.
[[0;32m OK [0m] Stopped [0;1;39mDiscard unused blocks once a week[0m.
[[0;32m OK [0m] Stopped [0;1;39mRefresh fwupd metadata regularly[0m.
[[0;32m OK [0m] Stopped [0;1;39mDaily rotation of log files[0m.
[[0;32m OK [0m] Stopped [0;1;39mDaily man-db regeneration[0m.
[[0;32m OK [0m] Stopped [0;1;39mMessage of the Day[0m.
[[0;32m OK [0m] Stopped [0;1;39mDaily Cleanup of Temporary Directories[0m.
[[0;32m OK [0m] Stopped [0;1;39mUbuntu Advantage Timer for running repeated jobs[0m.
[[0;32m OK [0m] Stopped [0;1;39mDownload data for …ailed at package install time[0m.
[[0;32m OK [0m] Stopped [0;1;39mCheck to see wheth…w version of Ubuntu available[0m.
[[0;32m OK [0m] Closed [0;1;39mLVM2 poll daemon socket[0m.
[[0;32m OK [0m] Closed [0;1;39mLoad/Save RF Kill Switch Status /dev/rfkill Watch[0m.
Stopping [0;1;39mModem Manager[0m...
Stopping [0;1;39mLSB: automatic crash report generation[0m...
Stopping [0;1;39mAutomounts filesystems on demand[0m...
Stopping [0;1;39mAvailability of block devices[0m...
Stopping [0;1;39mRegular background program processing daemon[0m...
Stopping [0;1;39mCreate final runt…dir for shutdown pivot root[0m...
Stopping [0;1;39mGetty on tty1[0m...
Stopping [0;1;39mirqbalance daemon[0m...
Stopping [0;1;39mKernel crash dump capture service[0m...
Stopping [0;1;39mLSB: Load kernel image with kexec[0m...
Stopping [0;1;39mMUNGE authentication service[0m...
Stopping [0;1;39mDispatcher daemon for systemd-networkd[0m...
Stopping [0;1;39mpNFS block layout mapping daemon[0m...
Stopping [0;1;39mNFS server and services[0m...
Stopping [0;1;39mPackageKit Daemon[0m...
[[0;32m OK [0m] Stopped [0;1;39mTerminate Plymouth Boot Screen[0m.
[[0;32m OK [0m] Stopped [0;1;39mPostfix Mail Transport Agent[0m.
Stopping [0;1;39mPostfix Mail Transport Agent (instance -)[0m...
Stopping [0;1;39mLSB: Start/Stop t…mote power manager) daemon.[0m...
Stopping [0;1;39mSystem Logging Service[0m...
Stopping [0;1;39mSerial Getty on ttyS0[0m...
Stopping [0;1;39mSamba SMB Daemon[0m...
Stopping [0;1;39mOpenBSD Secure Shell server[0m...
Stopping [0;1;39mLoad/Save Random Seed[0m...
Stopping [0;1;39mDisk Manager[0m...
Stopping [0;1;39mUnattended Upgrades Shutdown[0m...
Stopping [0;1;39mLSB: Start vmtouch daemon[0m...
Stopping [0;1;39mLSB: Starts or stops the xinetd daemon.[0m...
[ 4012.795608] nfsd: last server has exited, flushing export cache
[[0;32m OK [0m] Stopped [0;1;39mirqbalance daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mDispatcher daemon for systemd-networkd[0m.
[[0;32m OK [0m] Stopped [0;1;39mpNFS block layout mapping daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mDisk Manager[0m.
[[0;32m OK [0m] Stopped [0;1;39mModem Manager[0m.
[[0;32m OK [0m] Stopped [0;1;39mMUNGE authentication service[0m.
[[0;32m OK [0m] Stopped [0;1;39mOpenBSD Secure Shell server[0m.
[[0;32m OK [0m] Stopped [0;1;39mSamba SMB Daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mRegular background program processing daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mSerial Getty on ttyS0[0m.
[[0;32m OK [0m] Stopped [0;1;39mGetty on tty1[0m.
[[0;32m OK [0m] Stopped [0;1;39mPackageKit Daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mSystem Logging Service[0m.
[[0;32m OK [0m] Stopped [0;1;39mAvailability of block devices[0m.
[[0;32m OK [0m] Stopped [0;1;39mLoad/Save Random Seed[0m.
[[0;32m OK [0m] Stopped [0;1;39mUnattended Upgrades Shutdown[0m.
[[0;32m OK [0m] Stopped [0;1;39mSession 49 of User root[0m.
[[0;32m OK [0m] Stopped [0;1;39mNFS server and services[0m.
[[0;32m OK [0m] Stopped [0;1;39mPostfix Mail Transport Agent (instance -)[0m.
[[0;32m OK [0m] Stopped [0;1;39mLSB: automatic crash report generation[0m.
[[0;32m OK [0m] Stopped [0;1;39mLSB: Start vmtouch daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mLSB: Load kernel image with kexec[0m.
[[0;32m OK [0m] Stopped [0;1;39mLSB: Start/Stop th…remote power manager) daemon.[0m.
[[0;32m OK [0m] Stopped [0;1;39mLSB: Starts or stops the xinetd daemon.[0m.
[[0;32m OK [0m] Removed slice [0;1;39mSlice /system/getty[0m.
[[0;32m OK [0m] Removed slice [0;1;39mSlice /system/postfix[0m.
[[0;32m OK [0m] Removed slice [0;1;39mSlice /system/serial-getty[0m.
[[0;32m OK [0m] Stopped target [0;1;39mSystem Time Synchronized[0m.
[[0;32m OK [0m] Stopped target [0;1;39mSystem Time Set[0m.
Stopping [0;1;39mchrony, an NTP client/server[0m...
Stopping [0;1;39mLSB: Execute the …-e command to reboot system[0m...
Stopping [0;1;39mNFSv4 ID-name mapping service[0m...
Stopping [0;1;39mNFS Mount Daemon[0m...
Stopping [0;1;39mNFSv4 Client Tracking Daemon[0m...
Stopping [0;1;39mSamba NMB Daemon[0m...
[[0;32m OK [0m] Stopped [0;1;39mHold until boot process finishes up[0m.
Stopping [0;1;39mAuthorization Manager[0m...
Stopping [0;1;39mNFS status monitor for NFSv2/3 locking.[0m...
Stopping [0;1;39mUser Login Management[0m...
Stopping [0;1;39mUser Manager for UID 0[0m...
[[0;32m OK [0m] Stopped [0;1;39mNFSv4 ID-name mapping service[0m.
[[0;32m OK [0m] Stopped [0;1;39mNFSv4 Client Tracking Daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mchrony, an NTP client/server[0m.
[[0;32m OK [0m] Stopped [0;1;39mSamba NMB Daemon[0m.
[ 4013.313215] kdump-tools[12242]: Stopping kdump-tools:
[ 4013.315155] kdump-tools[12340]: * unloaded kdump kernel
[[0;32m OK [0m] Stopped [0;1;39mAuthorization Manager[0m.
[[0;32m OK [0m] Stopped [0;1;39mNFS Mount Daemon[0m.
[[0;32m OK [0m] Stopped [0;1;39mNFS status monitor for NFSv2/3 locking.[0m.
[[0;32m OK [0m] Stopped [0;1;39mUser Manager for UID 0[0m.
[[0;32m OK [0m] Stopped [0;1;39mKernel crash dump capture service[0m.
[[0;32m OK [0m] Stopped [0;1;39mUser Login Management[0m.
[[0;32m OK [0m] Stopped target [0;1;39mHost and Network Name Lookups[0m.
[[0;32m OK [0m] Stopped target [0;1;39mrpc_pipefs.target[0m.
Unmounting [0;1;39mRPC Pipe File System[0m...
Stopping [0;1;39mUser Runtime Directory /run/user/0[0m...
[[0;32m OK [0m] Unmounted [0;1;39m/run/user/0[0m.
[[0;32m OK [0m] Stopped [0;1;39mLSB: Execute the k…c -e command to reboot system[0m.
[[0;32m OK [0m] Unmounted [0;1;39mRPC Pipe File System[0m.
[[0;32m OK [0m] Stopped [0;1;39mUser Runtime Directory /run/user/0[0m.
[[0;32m OK [0m] Removed slice [0;1;39mUser Slice of UID 0[0m.
Stopping [0;1;39mPermit User Sessions[0m...
[[0;32m OK [0m] Stopped [0;1;39mPermit User Sessions[0m.
[[0;32m OK [0m] Stopped [0;1;39mAutomounts filesystems on demand[0m.
[[0;32m OK [0m] Stopped target [0;1;39mRemote File Systems[0m.
Unmounting [0;1;39m/autotest[0m...
Unmounting [0;1;39m/scratch[0m...
[[0;32m OK [0m] Unmounted [0;1;39m/autotest[0m.
[[0;32m OK [0m] Unmounted [0;1;39m/scratch[0m.
[[0;32m OK [0m] Stopped target [0;1;39mNetwork is Online[0m.
[[0;32m OK [0m] Stopped target [0;1;39mNetwork[0m.
[[0;32m OK [0m] Stopped target [0;1;39mPreparation for Remote File Systems[0m.
[[0;32m OK [0m] Stopped target [0;1;39mNFS client services[0m.
[[0;32m OK [0m] Stopped [0;1;39mNetwork Manager Wait Online[0m.
Stopping [0;1;39mNetwork Manager[0m...
Stopping [0;1;39mNetwork Name Resolution[0m...
Stopping [0;1;39mWPA supplicant[0m...
[[0;32m OK [0m] Stopped [0;1;39mNetwork Name Resolution[0m.
Stopping [0;1;39mNetwork Configuration[0m...
[[0;32m OK [0m] Stopped [0;1;39mCreate final runtime dir for shutdown pivot root[0m.
[[0;32m OK [0m] Stopped [0;1;39mNetwork Configuration[0m.
[[0;32m OK [0m] Stopped [0;1;39mNetwork Manager[0m.
[[0;32m OK [0m] Stopped target [0;1;39mPreparation for Network[0m.
[[0;32m OK [0m] Stopped [0;1;39mWPA supplicant[0m.
[[0;32m OK [0m] Stopped target [0;1;39mBasic System[0m.
[[0;32m OK [0m] Stopped target [0;1;39mPath Units[0m.
[[0;32m OK [0m] Stopped [0;1;39mTrigger to poll fo…y enabled on GCP LTS non-pro)[0m.
[[0;32m OK [0m] Stopped target [0;1;39mSlice Units[0m.
[[0;32m OK [0m] Removed slice [0;1;39mUser and Session Slice[0m.
[[0;32m OK [0m] Stopped target [0;1;39mSocket Units[0m.
[[0;32m OK [0m] Closed [0;1;39mOpen-iSCSI iscsid Socket[0m.
[[0;32m OK [0m] Closed [0;1;39mSyslog Socket[0m.
[[0;32m OK [0m] Closed [0;1;39mNetwork Service Netlink Socket[0m.
[[0;32m OK [0m] Closed [0;1;39mUUID daemon activation socket[0m.
[[0;32m OK [0m] Stopped target [0;1;39mSystem Initialization[0m.
[[0;32m OK [0m] Stopped target [0;1;39mLocal Encrypted Volumes[0m.
[[0;32m OK [0m] Stopped [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[[0;32m OK [0m] Stopped [0;1;39mForward Password R…uests to Wall Directory Watch[0m.
[[0;32m OK [0m] Stopped target [0;1;39mSwaps[0m.
[[0;32m OK [0m] Stopped target [0;1;39mLocal Verity Protected Volumes[0m.
Deactivating swap [0;1;39m/swap.img[0m...
Stopping [0;1;39mSet Up Additional Binary Formats[0m...
[[0;32m OK [0m] Stopped [0;1;39mApply Kernel Variables[0m.
[[0;32m OK [0m] Stopped [0;1;39mLoad Kernel Modules[0m.
Stopping [0;1;39mRecord System Boot/Shutdown in UTMP[0m...
[[0;32m OK [0m] Stopped [0;1;39mSet Up Additional Binary Formats[0m.
[[0;32m OK [0m] Unset automount [0;1;39mArbitrary …s File System Automount Point[0m.
[[0;32m OK [0m] Stopped [0;1;39mRecord System Boot/Shutdown in UTMP[0m.
[[0;32m OK [0m] Stopped [0;1;39mCreate Volatile Files and Directories[0m.
[[0;32m OK [0m] Stopped target [0;1;39mLocal File Systems[0m.
Unmounting [0;1;39m/run/credentials/systemd-sysusers.service[0m...
[[0;32m OK [0m] Unmounted [0;1;39m/run/credentials/systemd-sysusers.service[0m.
[[0;32m OK [0m] Stopped target [0;1;39mPreparation for Local File Systems[0m.
Stopping [0;1;39mMonitoring of LVM…meventd or progress polling[0m...
Stopping [0;1;39mDevice-Mapper Multipath Device Controller[0m...
[[0;32m OK [0m] Stopped [0;1;39mCreate Static Device Nodes in /dev[0m.
[[0;32m OK [0m] Stopped [0;1;39mCreate System Users[0m.
[[0;32m OK [0m] Deactivated swap [0;1;39m/swap.img[0m.
[[0;32m OK [0m] Reached target [0;1;39mUnmount All Filesystems[0m.
[[0;32m OK [0m] Stopped [0;1;39mMonitoring of LVM2… dmeventd or progress polling[0m.
[[0;32m OK [0m] Stopped [0;1;39mDevice-Mapper Multipath Device Controller[0m.
[[0;32m OK [0m] Stopped [0;1;39mRemount Root and Kernel File Systems[0m.
[[0;32m OK [0m] Reached target [0;1;39mSystem Shutdown[0m.
[[0;32m OK [0m] Reached target [0;1;39mLate Shutdown Services[0m.
[[0;32m OK [0m] Finished [0;1;39mSystem Reboot[0m.
[[0;32m OK [0m] Reached target [0;1;39mSystem Reboot[0m.
[ 4013.777666] systemd-shutdown[1]: Syncing filesystems and block devices.
[ 4013.928904] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
[ 4013.933103] systemd-journald[354]: Received SIGTERM from PID 1 (systemd-shutdow).
[ 4014.050704] systemd-shutdown[1]: Sending SIGKILL to remaining processes...
[ 4014.054422] systemd-shutdown[1]: Unmounting file systems.
[ 4014.056077] [12891]: Remounting '/' read-only in with options '(null)'.
[ 4014.130080] EXT4-fs (vda1): re-mounted. Opts: (null). Quota mode: none.
[ 4014.132785] systemd-shutdown[1]: All filesystems unmounted.
[ 4014.133792] systemd-shutdown[1]: Deactivating swaps.
[ 4014.134728] systemd-shutdown[1]: All swaps deactivated.
[ 4014.135659] systemd-shutdown[1]: Detaching loop devices.
[ 4014.137346] systemd-shutdown[1]: All loop devices detached.
[ 4014.138337] systemd-shutdown[1]: Stopping MD devices.
[ 4014.139277] systemd-shutdown[1]: All MD devices stopped.
[ 4014.140240] systemd-shutdown[1]: Detaching DM devices.
[ 4014.141209] systemd-shutdown[1]: All DM devices detached.
[ 4014.142152] systemd-shutdown[1]: All filesystems, swaps, loop devices, MD devices and DM devices detached.
[ 4014.164077] systemd-shutdown[1]: Successfully changed into root pivot.
[ 4014.165219] systemd-shutdown[1]: Returning to initrd...
[ 4014.173299] shutdown[1]: Syncing filesystems and block devices.
[ 4014.174377] shutdown[1]: Sending SIGTERM to remaining processes...
[ 4014.176823] shutdown[1]: Sending SIGKILL to remaining processes...
[ 4014.179129] shutdown[1]: Unmounting file systems.
[ 4014.180633] sd-umoun[12897]: Unmounting '/oldroot/sys/fs/fuse/connections'.
[ 4014.182463] sd-umoun[12898]: Unmounting '/oldroot/sys/kernel/config'.
[ 4014.184170] sd-remou[12899]: Remounting '/oldroot/proc/fs/nfsd' read-only in with options '(null)'.
[ 4014.186262] sd-umoun[12900]: Unmounting '/oldroot/proc/fs/nfsd'.
[ 4014.187926] sd-umoun[12901]: Unmounting '/oldroot/sys/kernel/tracing'.
[ 4014.189647] sd-umoun[12902]: Unmounting '/oldroot/sys/kernel/debug'.
[ 4014.191298] sd-umoun[12903]: Unmounting '/oldroot/dev/mqueue'.
[ 4014.192878] sd-umoun[12904]: Unmounting '/oldroot/dev/hugepages'.
[ 4014.194546] sd-umoun[12905]: Unmounting '/oldroot/sys/fs/bpf'.
[ 4014.196192] sd-umoun[12906]: Unmounting '/oldroot/sys/fs/pstore'.
[ 4014.197807] sd-umoun[12907]: Unmounting '/oldroot/sys/fs/cgroup'.
[ 4014.199526] sd-umoun[12908]: Unmounting '/oldroot/run/lock'.
[ 4014.201162] sd-umoun[12909]: Unmounting '/oldroot/dev/shm'.
[ 4014.202711] sd-umoun[12910]: Unmounting '/oldroot/sys/kernel/security'.
[ 4014.204391] sd-umoun[12911]: Unmounting '/oldroot'.
[ 4014.205285] sd-umoun[12911]: Failed to unmount /oldroot: Device or resource busy
[ 4014.207065] sd-umoun[12912]: Unmounting '/oldroot/run'.
[ 4014.208548] sd-umoun[12913]: Unmounting '/oldroot/dev/pts'.
[ 4014.210119] sd-umoun[12914]: Unmounting '/oldroot/dev'.
[ 4014.211653] sd-umoun[12915]: Unmounting '/oldroot/proc'.
[ 4014.213147] sd-umoun[12916]: Unmounting '/oldroot/sys'.
[ 4014.214741] sd-umoun[12917]: Unmounting '/oldroot'.
[ 4014.434532] shutdown[1]: All filesystems unmounted.
[ 4014.435501] shutdown[1]: Deactivating swaps.
[ 4014.436343] shutdown[1]: All swaps deactivated.
[ 4014.437162] shutdown[1]: Detaching loop devices.
[ 4014.438745] shutdown[1]: All loop devices detached.
[ 4014.439660] shutdown[1]: Stopping MD devices.
[ 4014.440508] shutdown[1]: All MD devices stopped.
[ 4014.441332] shutdown[1]: Detaching DM devices.
[ 4014.442172] shutdown[1]: All DM devices detached.
[ 4014.443000] shutdown[1]: All filesystems, swaps, loop devices, MD devices and DM devices detached.
[ 4014.449619] shutdown[1]: Syncing filesystems and block devices.
[ 4014.450706] shutdown[1]: Rebooting.
[ 4014.452689] reboot: Restarting system
[ 4014.453362] reboot: machine restart
[1;256r[256;256H[6n
Google, Inc.
Serial Graphics Adapter 06/09/14
SGABIOS $Id: sgabios.S 8 2010-04-22 00:03:40Z nlaredo $ (mockbuild@) Mon Jun 9 21:33:48 UTC 2014
Term: 80x24
4 0
[2J
SeaBIOS (version 1.11.0-2.el7)
Machine UUID 744bd410-641c-4b44-89e6-a7d1cc9fda86
iPXE (http://ipxe.org) 00:03.0 C100 PCI2.10 PnP PMM+BFF945D0+BFED45D0 C100
Press Ctrl-B to configure iPXE (PCI 00:03.0)...
Booting from ROM...
iPXE (PCI 00:03.0) starting execution...ok
iPXE initialising devices...ok
[40;37;1m
i[m
i[40;37;1mP[mP[40;37;1mX[mX[40;37;1mE[mE[40;37;1m [m [40;37;1m1[m1[40;37;1m.[m.[40;37;1m0[m0[40;37;1m.[m.[40;37;1m0[m0[40;37;1m+[m+[40;37;1m [m [40;37;1m([m([40;37;1m1[m1[40;37;1m3[m3[40;37;1m3[m3[40;37;1mf[mf[40;37;1m4[m4[40;37;1mc[mc[40;37;1m)[m) -- Open Source Network Boot Firmware -- [40;36;22mh[mh[40;36;22mt[mt[40;36;22mt[mt[40;36;22mp[mp[40;36;22m:[m:[40;36;22m/[m/[40;36;22m/[m/[40;36;22mi[mi[40;36;22mp[mp[40;36;22mx[mx[40;36;22me[me[40;36;22m.[m.[40;36;22mo[mo[40;36;22mr[mr[40;36;22mg[mg
Features: DNS HTTP iSCSI TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT
Press Ctrl-B for the iPXE command line...
net0: 52:54:00:3e:d9:7f using virtio-net on 0000:00:03.0 (open)
[Link:up, TX:0 TXE:0 RX:0 RXE:0]
Configuring (net0 52:54:00:3e:d9:7f).............. ok
net0: 10.240.44.153/255.255.240.0 gw 10.240.32.1
Next server: 10.240.32.5
Filename: /pxelinux.0
tftp://10.240.32.5//pxelinux.0... ok
pxelinux.0 : 26759 bytes [PXE-NBP]
[1S
PXELINUX 4.05 0x5bd8f633 Copyright (C) 1994-2011 H. Peter Anvin et al[1S
!PXE entry point found (we hope) at 9C2C:0160 via plan A[1S
UNDI code segment at 9C2C len 0802[1S
UNDI data segment at 9CB0 len 2CE0[1S
Getting cached packet 01 02 03[1S
My IP address seems to be 0AF02C99 10.240.44.153[1S
ip=10.240.44.153:10.240.32.5:10.240.32.1:255.255.240.0[1S
BOOTIF=01-52-54-00-3e-d9-7f[1S
SYSUUID=10d44b74-1c64-444b-89e6-a7d1cc9fda86[1S
TFTP prefix: /[1S
Trying to load: pxelinux.cfg/10d44b74-1c64-444b-89e6-a7d1cc9fda86
Trying to load: pxelinux.cfg/01-52-54-00-3e-d9-7f ok[1S
Booting from local disk...[1S
No more network devices
Press Ctrl-B for the iPXE command line...
Booting from Hard Disk...
Loading Linux 5.15.0-94-generic ...
Loading initial ramdisk ...
[ 0.000000] Linux version 5.15.0-94-generic (buildd@lcy02-amd64-096) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 (Ubuntu 5.15.0-94.104-generic 5.15.136)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=a72e3805-c7e8-4bb3-840b-92bd865f2f09 ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 slab_nomerge slub_nomerge crashkernel=512M-:296M
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Hygon HygonGenuine
[ 0.000000] Centaur CentaurHauls
[ 0.000000] zhaoxin Shanghai
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfff9fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000bfffa000-0x00000000bfffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.4 present.
[ 0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr 70001001, primary cpu clock
[ 0.000004] kvm-clock: using sched offset of 4057797315120 cycles
[ 0.000010] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000022] tsc: Detected 2294.686 MHz processor
[ 0.000298] last_pfn = 0xbfffa max_arch_pfn = 0x400000000
[ 0.000347] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.012221] found SMP MP-table at [mem 0x000f6390-0x000f639f]
[ 0.012265] Using GB pages for direct mapping
[ 0.012409] RAMDISK: [mem 0x2b8bf000-0x31c56fff]
[ 0.012444] ACPI: Early table checksum verification disabled
[ 0.012474] ACPI: RSDP 0x00000000000F61C0 000014 (v00 BOCHS )
[ 0.012488] ACPI: RSDT 0x00000000BFFFFB40 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.012505] ACPI: FACP 0x00000000BFFFF177 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.012517] ACPI: DSDT 0x00000000BFFFE040 001137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.012527] ACPI: FACS 0x00000000BFFFE000 000040
[ 0.012532] ACPI: SSDT 0x00000000BFFFF1EB 0008D5 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.012537] ACPI: APIC 0x00000000BFFFFAC0 000080 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.012541] ACPI: Reserving FACP table memory at [mem 0xbffff177-0xbffff1ea]
[ 0.012543] ACPI: Reserving DSDT table memory at [mem 0xbfffe040-0xbffff176]
[ 0.012545] ACPI: Reserving FACS table memory at [mem 0xbfffe000-0xbfffe03f]
[ 0.012546] ACPI: Reserving SSDT table memory at [mem 0xbffff1eb-0xbffffabf]
[ 0.012548] ACPI: Reserving APIC table memory at [mem 0xbffffac0-0xbffffb3f]
[ 0.013053] No NUMA configuration found
[ 0.013056] Faking a node at [mem 0x0000000000000000-0x00000000bfff9fff]
[ 0.013076] NODE_DATA(0) allocated [mem 0xbffd0000-0xbfff9fff]
[ 0.013594] Reserving 296MB of memory at 2768MB for crashkernel (System RAM: 3071MB)
[ 0.013638] Zone ranges:
[ 0.013647] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.013652] DMA32 [mem 0x0000000001000000-0x00000000bfff9fff]
[ 0.013658] Normal empty
[ 0.013660] Device empty
[ 0.013662] Movable zone start for each node
[ 0.013667] Early memory node ranges
[ 0.013669] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.013671] node 0: [mem 0x0000000000100000-0x00000000bfff9fff]
[ 0.013673] Initmem setup node 0 [mem 0x0000000000001000-0x00000000bfff9fff]
[ 0.013694] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.013909] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.053926] On node 0, zone DMA32: 6 pages in unavailable ranges
[ 0.054468] ACPI: PM-Timer IO Port: 0x608
[ 0.054499] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.054551] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.054556] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.054558] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.054565] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.054570] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.054572] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.054577] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.054584] TSC deadline timer available
[ 0.054586] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.054626] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.054628] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.054630] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[ 0.054631] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[ 0.054637] [mem 0xc0000000-0xfeffbfff] available for PCI devices
[ 0.054638] Booting paravirtualized kernel on KVM
[ 0.054647] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.054662] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.056541] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u1048576
[ 0.056580] kvm-guest: stealtime: cpu 0, msr bfc33080
[ 0.056584] kvm-guest: PV spinlocks enabled
[ 0.056587] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)
[ 0.056606] Built 1 zonelists, mobility grouping on. Total pages: 773882
[ 0.056608] Policy zone: DMA32
[ 0.056610] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=a72e3805-c7e8-4bb3-840b-92bd865f2f09 ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 slab_nomerge slub_nomerge crashkernel=512M-:296M
[ 0.056980] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic biosdevname=0", will be passed to user space.
[ 0.060446] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.061972] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.062048] mem auto-init: stack:off, heap alloc:on, heap free:off
[ 0.070058] Memory: 2625904K/3145312K available (16393K kernel code, 4394K rwdata, 10876K rodata, 3356K init, 18716K bss, 519148K reserved, 0K cma-reserved)
[ 0.070471] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.070507] Kernel/User page tables isolation: enabled
[ 0.070598] ftrace: allocating 50666 entries in 198 pages
[ 0.101327] ftrace: allocated 198 pages with 4 groups
[ 0.101685] rcu: Hierarchical RCU implementation.
[ 0.101688] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=2.
[ 0.101690] Rude variant of Tasks RCU enabled.
[ 0.101691] Tracing variant of Tasks RCU enabled.
[ 0.101695] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.101696] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.107319] NR_IRQS: 524544, nr_irqs: 440, preallocated irqs: 16
[ 0.107532] random: crng init done
[ 0.108472] Console: colour VGA+ 80x25
[ 0.119058] printk: console [tty0] enabled
[ 0.222039] printk: console [ttyS0] enabled
[ 0.222796] ACPI: Core revision 20210730
[ 0.223618] APIC: Switch to symmetric I/O mode setup
[ 0.224806] x2apic enabled
[ 0.225688] Switched APIC routing to physical x2apic.
[ 0.228011] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.229204] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x211399552f8, max_idle_ns: 440795292447 ns
[ 0.231125] Calibrating delay loop (skipped) preset value.. 4589.37 BogoMIPS (lpj=9178744)
[ 0.235240] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.236265] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.237397] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.239123] Spectre V2 : Mitigation: Retpolines
[ 0.240023] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.241573] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.243122] Speculative Store Bypass: Vulnerable
[ 0.244030] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[ 0.245259] TAA: Vulnerable: Clear CPU buffers attempted, no microcode
[ 0.247121] MMIO Stale Data: Unknown: No mitigations
[ 0.248087] SRBDS: Unknown: Dependent on hypervisor status
[ 0.249152] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.250597] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.251121] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.252316] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.253485] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.276481] Freeing SMP alternatives memory: 44K
[ 0.277424] pid_max: default: 32768 minimum: 301
[ 0.278363] LSM: Security Framework initializing
[ 0.279139] landlock: Up and running.
[ 0.279889] Yama: becoming mindful.
[ 0.280682] AppArmor: AppArmor initialized
[ 0.281600] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.283179] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.285206] smpboot: CPU0: Intel Core Processor (Broadwell) (family: 0x6, model: 0x3d, stepping: 0x2)
[ 0.287120] Performance Events: unsupported p6 CPU model 61 no PMU driver, software events only.
[ 0.287120] signal: max sigframe size: 1776
[ 0.287165] rcu: Hierarchical SRCU implementation.
[ 0.288806] NMI watchdog: Perf NMI watchdog permanently disabled
[ 0.290086] smp: Bringing up secondary CPUs ...
[ 0.291252] x86: Booting SMP configuration:
[ 0.292100] .... node #0, CPUs: #1
[ 0.123597] kvm-clock: cpu 1, msr 70001041, secondary cpu clock
[ 0.123597] smpboot: CPU 1 Converting physical 0 to logical die 1
[ 0.295141] kvm-guest: stealtime: cpu 1, msr bfd33080
[ 0.296032] smp: Brought up 1 node, 2 CPUs
[ 0.296032] smpboot: Max logical packages: 2
[ 0.296646] smpboot: Total of 2 processors activated (9178.74 BogoMIPS)
[ 0.299222] devtmpfs: initialized
[ 0.299820] x86/mm: Memory block size: 128MB
[ 0.301057] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.301057] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.303183] pinctrl core: initialized pinctrl subsystem
[ 0.304345] PM: RTC time: 13:42:34, date: 2024-12-16
[ 0.305467] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.307072] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[ 0.307595] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[ 0.309329] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[ 0.310637] audit: initializing netlink subsys (disabled)
[ 0.311172] audit: type=2000 audit(1734356556.684:1): state=initialized audit_enabled=0 res=1
[ 0.311326] thermal_sys: Registered thermal governor 'fair_share'
[ 0.312565] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.313597] thermal_sys: Registered thermal governor 'step_wise'
[ 0.314606] thermal_sys: Registered thermal governor 'user_space'
[ 0.315129] thermal_sys: Registered thermal governor 'power_allocator'
[ 0.316163] EISA bus registered
[ 0.317824] cpuidle: using governor ladder
[ 0.318553] cpuidle: using governor menu
[ 0.319253] ACPI: bus type PCI registered
[ 0.319966] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 0.321280] PCI: Using configuration type 1 for base access
[ 0.323857] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.325375] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.325375] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.327221] ACPI: Added _OSI(Module Device)
[ 0.328003] ACPI: Added _OSI(Processor Device)
[ 0.331132] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.331954] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.332863] ACPI: Added _OSI(Linux-Dell-Video)
[ 0.333636] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[ 0.334522] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[ 0.336644] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.338590] ACPI: Interpreter enabled
[ 0.339142] ACPI: PM: (supports S0 S5)
[ 0.339817] ACPI: Using IOAPIC for interrupt routing
[ 0.340682] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.342143] PCI: Using E820 reservations for host bridge windows
[ 0.343404] ACPI: Enabled 16 GPEs in block 00 to 0F
[ 0.347369] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.348429] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[ 0.349772] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 0.351493] acpiphp: Slot [3] registered
[ 0.352227] acpiphp: Slot [4] registered
[ 0.352952] acpiphp: Slot [5] registered
[ 0.353669] acpiphp: Slot [6] registered
[ 0.354400] acpiphp: Slot [7] registered
[ 0.355137] acpiphp: Slot [8] registered
[ 0.355867] acpiphp: Slot [9] registered
[ 0.356582] acpiphp: Slot [10] registered
[ 0.357320] acpiphp: Slot [11] registered
[ 0.358058] acpiphp: Slot [12] registered
[ 0.358793] acpiphp: Slot [13] registered
[ 0.359152] acpiphp: Slot [14] registered
[ 0.359888] acpiphp: Slot [15] registered
[ 0.360633] acpiphp: Slot [16] registered
[ 0.361374] acpiphp: Slot [17] registered
[ 0.362104] acpiphp: Slot [18] registered
[ 0.362844] acpiphp: Slot [19] registered
[ 0.363152] acpiphp: Slot [20] registered
[ 0.363892] acpiphp: Slot [21] registered
[ 0.364624] acpiphp: Slot [22] registered
[ 0.365356] acpiphp: Slot [23] registered
[ 0.366095] acpiphp: Slot [24] registered
[ 0.366820] acpiphp: Slot [25] registered
[ 0.367152] acpiphp: Slot [26] registered
[ 0.367899] acpiphp: Slot [27] registered
[ 0.368630] acpiphp: Slot [28] registered
[ 0.369352] acpiphp: Slot [29] registered
[ 0.370085] acpiphp: Slot [30] registered
[ 0.370819] acpiphp: Slot [31] registered
[ 0.371139] PCI host bridge to bus 0000:00
[ 0.371856] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.372780] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.373893] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.375009] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 0.375124] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[ 0.376457] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[ 0.378325] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[ 0.379925] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[ 0.386482] pci 0000:00:01.1: reg 0x20: [io 0xc0a0-0xc0af]
[ 0.389388] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.390586] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.391123] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.392318] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.393691] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
[ 0.399693] pci 0000:00:01.2: reg 0x20: [io 0xc040-0xc05f]
[ 0.403293] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[ 0.404906] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.406118] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.407940] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
[ 0.411124] pci 0000:00:03.0: reg 0x10: [io 0xc060-0xc07f]
[ 0.414228] pci 0000:00:03.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff]
[ 0.425251] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
[ 0.427181] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000
[ 0.430450] pci 0000:00:04.0: reg 0x10: [io 0xc000-0xc03f]
[ 0.433272] pci 0000:00:04.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff]
[ 0.444973] pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00
[ 0.447124] pci 0000:00:05.0: reg 0x10: [io 0xc080-0xc09f]
[ 0.460407] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[ 0.461571] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[ 0.462727] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
[ 0.463247] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[ 0.464323] ACPI: PCI: Interrupt link LNKS configured for IRQ 9
[ 0.465622] iommu: Default domain type: Translated
[ 0.465622] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.467375] SCSI subsystem initialized
[ 0.468115] vgaarb: loaded
[ 0.468115] ACPI: bus type USB registered
[ 0.468450] usbcore: registered new interface driver usbfs
[ 0.469394] usbcore: registered new interface driver hub
[ 0.471132] usbcore: registered new device driver usb
[ 0.472026] pps_core: LinuxPPS API ver. 1 registered
[ 0.472873] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.474353] PTP clock support registered
[ 0.475130] EDAC MC: Ver: 3.0.0
[ 0.475986] NetLabel: Initializing
[ 0.475986] NetLabel: domain hash size = 128
[ 0.476742] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO
[ 0.477714] NetLabel: unlabeled traffic allowed by default
[ 0.479150] PCI: Using ACPI for IRQ routing
[ 0.482076] clocksource: Switched to clocksource kvm-clock
[ 0.505378] VFS: Disk quotas dquot_6.6.0
[ 0.506135] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.507461] AppArmor: AppArmor Filesystem Enabled
[ 0.508295] pnp: PnP ACPI init
[ 0.509368] pnp: PnP ACPI: found 5 devices
[ 0.517037] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.518564] NET: Registered PF_INET protocol family
[ 0.519882] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.523844] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.525502] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.527000] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.528802] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[ 0.530051] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.531628] MPTCP token hash table entries: 4096 (order: 4, 98304 bytes, linear)
[ 0.533608] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.534772] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.535994] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.536956] NET: Registered PF_XDP protocol family
[ 0.537800] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.538828] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.539858] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[ 0.541006] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
[ 0.542183] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.543165] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.544179] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.545547] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[ 0.547101] PCI: CLS 0 bytes, default 64
[ 0.547944] Trying to unpack rootfs image as initramfs...
[ 0.555725] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x211399552f8, max_idle_ns: 440795292447 ns
[ 0.564403] Initialise system trusted keyrings
[ 0.565240] Key type blacklist registered
[ 0.566009] workingset: timestamp_bits=36 max_order=20 bucket_order=0
[ 0.569059] zbud: loaded
[ 0.570119] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.571681] fuse: init (API version 7.34)
[ 0.572887] integrity: Platform Keyring initialized
[ 0.587653] Key type asymmetric registered
[ 0.588408] Asymmetric key parser 'x509' registered
[ 0.589289] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
[ 0.590586] io scheduler mq-deadline registered
[ 0.591697] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 0.592938] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.594216] ACPI: button: Power Button [PWRF]
[ 0.596070] ACPI: \_SB_.LNKC: Enabled at IRQ 10
[ 0.596954] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
[ 0.598395] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[ 0.600425] ACPI: \_SB_.LNKA: Enabled at IRQ 10
[ 0.601264] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[ 0.603379] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[ 0.627153] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 0.631551] Linux agpgart interface v0.103
[ 0.635971] loop: module loaded
[ 0.643720] scsi host0: ata_piix
[ 0.647697] scsi host1: ata_piix
[ 0.648358] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
[ 0.649484] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
[ 0.651080] tun: Universal TUN/TAP device driver, 1.6
[ 0.652315] PPP generic driver version 2.4.2
[ 0.653175] VFIO - User Level meta-driver version: 0.3
[ 0.654162] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.655269] ehci-pci: EHCI PCI platform driver
[ 0.656087] ehci-platform: EHCI generic platform driver
[ 0.657000] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.658045] ohci-pci: OHCI PCI platform driver
[ 0.658837] ohci-platform: OHCI generic platform driver
[ 0.659754] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.661931] uhci_hcd 0000:00:01.2: UHCI Host Controller
[ 0.662868] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
[ 0.664132] uhci_hcd 0000:00:01.2: detected 2 ports
[ 0.665108] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
[ 0.666207] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 5.15
[ 0.667621] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 0.668853] usb usb1: Product: UHCI Host Controller
[ 0.669722] usb usb1: Manufacturer: Linux 5.15.0-94-generic uhci_hcd
[ 0.670802] usb usb1: SerialNumber: 0000:00:01.2
[ 0.671995] hub 1-0:1.0: USB hub found
[ 0.672718] hub 1-0:1.0: 2 ports detected
[ 0.673684] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[ 0.675975] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.676871] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.678215] mousedev: PS/2 mouse device common for all mice
[ 0.679381] rtc_cmos 00:00: RTC can wake from S4
[ 0.680756] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[ 0.682935] rtc_cmos 00:00: registered as rtc0
[ 0.683872] rtc_cmos 00:00: setting system clock to 2024-12-16T13:42:35 UTC (1734356555)
[ 0.685286] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
[ 0.686344] i2c_dev: i2c /dev entries driver
[ 0.687140] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[ 0.689143] device-mapper: uevent: version 1.0.3
[ 0.690079] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[ 0.691538] platform eisa.0: Probing EISA bus 0
[ 0.692337] platform eisa.0: EISA: Cannot allocate resource for mainboard
[ 0.693491] platform eisa.0: Cannot allocate resource for EISA slot 1
[ 0.694580] platform eisa.0: Cannot allocate resource for EISA slot 2
[ 0.695671] platform eisa.0: Cannot allocate resource for EISA slot 3
[ 0.696753] platform eisa.0: Cannot allocate resource for EISA slot 4
[ 0.697847] platform eisa.0: Cannot allocate resource for EISA slot 5
[ 0.698934] platform eisa.0: Cannot allocate resource for EISA slot 6
[ 0.700043] platform eisa.0: Cannot allocate resource for EISA slot 7
[ 0.701130] platform eisa.0: Cannot allocate resource for EISA slot 8
[ 0.702205] platform eisa.0: EISA: Detected 0 cards
[ 0.703058] intel_pstate: CPU model not supported
[ 0.703916] ledtrig-cpu: registered to indicate activity on CPUs
[ 0.705023] drop_monitor: Initializing network drop monitor service
[ 0.706266] NET: Registered PF_INET6 protocol family
[ 1.538297] Freeing initrd memory: 101984K
[ 1.545190] Segment Routing with IPv6
[ 1.545938] In-situ OAM (IOAM) with IPv6
[ 1.546681] NET: Registered PF_PACKET protocol family
[ 1.547641] Key type dns_resolver registered
[ 1.548724] IPI shorthand broadcast: enabled
[ 1.549523] sched_clock: Marking stable (1429033898, 119597479)->(1586056850, -37425473)
[ 1.551031] registered taskstats version 1
[ 1.551872] Loading compiled-in X.509 certificates
[ 1.553481] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[ 1.555715] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969'
[ 1.557908] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19'
[ 1.559558] blacklist: Loading compiled-in revocation X.509 certificates
[ 1.560705] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing: 61482aa2830d0ab2ad5af10b7250da9033ddcef0'
[ 1.562304] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2017): 242ade75ac4a15e50d50c84b0d45ff3eae707a03'
[ 1.563997] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (ESM 2018): 365188c1d374d6b07c3c8f240f8ef722433d6a8b'
[ 1.565735] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2019): c0746fd6c5da3ae827864651ad66ae47fe24b3e8'
[ 1.567417] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d'
[ 1.569139] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v2): 4cf046892d6fd3c9a5b03f98d845f90851dc6a8c'
[ 1.570854] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v3): 100437bb6de6e469b581e61cd66bce3ef4ed53af'
[ 1.572578] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9'
[ 1.574752] zswap: loaded using pool lzo/zbud
[ 1.575722] Key type .fscrypt registered
[ 1.576432] Key type fscrypt-provisioning registered
[ 1.579934] Key type encrypted registered
[ 1.580676] AppArmor: AppArmor sha1 policy hashing enabled
[ 1.581616] ima: No TPM chip found, activating TPM-bypass!
[ 1.582540] Loading compiled-in module X.509 certificates
[ 1.584041] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[ 1.585625] ima: Allocated hash algorithm: sha1
[ 1.586406] ima: No architecture policies found
[ 1.587200] evm: Initialising EVM extended attributes:
[ 1.588085] evm: security.selinux
[ 1.588695] evm: security.SMACK64
[ 1.589297] evm: security.SMACK64EXEC
[ 1.589956] evm: security.SMACK64TRANSMUTE
[ 1.590678] evm: security.SMACK64MMAP
[ 1.591331] evm: security.apparmor
[ 1.591957] evm: security.ima
[ 1.592536] evm: security.capability
[ 1.593175] evm: HMAC attrs: 0x1
[ 1.594049] PM: Magic number: 4:303:736
[ 1.594989] RAS: Correctable Errors collector initialized.
[ 1.597759] Freeing unused decrypted memory: 2036K
[ 1.600060] Freeing unused kernel image (initmem) memory: 3356K
[ 1.615617] Write protecting the kernel read-only data: 30720k
[ 1.618633] Freeing unused kernel image (text/rodata gap) memory: 2036K
[ 1.621071] Freeing unused kernel image (rodata/data gap) memory: 1412K
[ 1.681819] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 1.682931] x86/mm: Checking user space page tables
[ 1.736712] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 1.737816] Run /init as init process
Loading, please wait...
Starting version 249.11-0ubuntu3.12
[ 1.841780] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
[ 1.843824] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
[ 1.845483] virtio_blk virtio1: [vda] 209715200 512-byte logical blocks (107 GB/100 GiB)
[ 1.845577] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
[ 1.851400] vda: vda1 vda4
[ 1.851654] cryptd: max_cpu_qlen set to 1000
[ 1.858148] AVX2 version of gcm_enc/dec engaged.
[ 1.859054] AES CTR mode by8 optimization enabled
[ 1.864234] FDC 0 is a S82078B
Begin: Loading essential drivers ... [ 3.143603] raid6: avx2x4 gen() 26807 MB/s
[ 3.211604] raid6: avx2x4 xor() 10095 MB/s
[ 3.279603] raid6: avx2x2 gen() 25392 MB/s
[ 3.347603] raid6: avx2x2 xor() 15849 MB/s
[ 3.415604] raid6: avx2x1 gen() 23801 MB/s
[ 3.483603] raid6: avx2x1 xor() 13922 MB/s
[ 3.551603] raid6: sse2x4 gen() 14575 MB/s
[ 3.619604] raid6: sse2x4 xor() 8394 MB/s
[ 3.687604] raid6: sse2x2 gen() 13882 MB/s
[ 3.755604] raid6: sse2x2 xor() 9163 MB/s
[ 3.823604] raid6: sse2x1 gen() 12237 MB/s
[ 3.891603] raid6: sse2x1 xor() 8168 MB/s
[ 3.892367] raid6: using algorithm avx2x4 gen() 26807 MB/s
[ 3.893308] raid6: .... xor() 10095 MB/s, rmw enabled
[ 3.894176] raid6: using avx2x2 recovery algorithm
[ 3.896283] xor: automatically using best checksumming function avx
[ 3.898530] async_tx: api initialized (async)
done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... [ 3.966785] Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes
Scanning for Btrfs filesystems
done.
Begin: Will now check root file system ... fsck from util-linux 2.37.2
[/usr/sbin/fsck.ext4 (1) -- /dev/vda1] fsck.ext4 -a -C0 /dev/vda1
/dev/vda1: clean, 149724/1310720 files, 2465999/5242880 blocks
done.
[ 4.014436] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[ 4.159198] systemd[1]: Inserted module 'autofs4'
[ 4.168716] systemd[1]: systemd 249.11-0ubuntu3.12 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 4.173650] systemd[1]: Detected virtualization kvm.
[ 4.174533] systemd[1]: Detected architecture x86-64.
Welcome to [1mUbuntu 22.04 LTS[0m!
[ 4.176673] systemd[1]: Hostname set to <trevis-106vm14.trevis.whamcloud.com>.
[ 4.261114] systemd[1]: Configuration file /run/systemd/system/netplan-ovs-cleanup.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
[ 4.323578] systemd[1]: Queued start job for default target Graphical Interface.
[ 4.325914] systemd[1]: Created slice Slice /system/modprobe.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/modprobe[0m.
[ 4.328052] systemd[1]: Created slice Slice /system/postfix.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/postfix[0m.
[ 4.330075] systemd[1]: Created slice Slice /system/serial-getty.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/serial-getty[0m.
[ 4.332123] systemd[1]: Created slice User and Session Slice.
[[0;32m OK [0m] Created slice [0;1;39mUser and Session Slice[0m.
[ 4.333921] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[[0;32m OK [0m] Started [0;1;39mForward Password R…uests to Wall Directory Watch[0m.
[ 4.336310] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[[0;32m OK [0m] Set up automount [0;1;39mArbitrary…s File System Automount Point[0m.
[ 4.338885] systemd[1]: Reached target Slice Units.
[[0;32m OK [0m] Reached target [0;1;39mSlice Units[0m.
[ 4.340418] systemd[1]: Reached target System Time Set.
[[0;32m OK [0m] Reached target [0;1;39mSystem Time Set[0m.
[ 4.342053] systemd[1]: Reached target Local Verity Protected Volumes.
[[0;32m OK [0m] Reached target [0;1;39mLocal Verity Protected Volumes[0m.
[ 4.344029] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[[0;32m OK [0m] Listening on [0;1;39mDevice-mapper event daemon FIFOs[0m.
[ 4.346061] systemd[1]: Listening on LVM2 poll daemon socket.
[[0;32m OK [0m] Listening on [0;1;39mLVM2 poll daemon socket[0m.
[ 4.347927] systemd[1]: Listening on multipathd control socket.
[[0;32m OK [0m] Listening on [0;1;39mmultipathd control socket[0m.
[ 4.353117] systemd[1]: Listening on RPCbind Server Activation Socket.
[[0;32m OK [0m] Listening on [0;1;39mRPCbind Server Activation Socket[0m.
[ 4.355347] systemd[1]: Listening on Syslog Socket.
[[0;32m OK [0m] Listening on [0;1;39mSyslog Socket[0m.
[ 4.356999] systemd[1]: Listening on fsck to fsckd communication Socket.
[[0;32m OK [0m] Listening on [0;1;39mfsck to fsckd communication Socket[0m.
[ 4.359044] systemd[1]: Listening on initctl Compatibility Named Pipe.
[[0;32m OK [0m] Listening on [0;1;39minitctl Compatibility Named Pipe[0m.
[ 4.361150] systemd[1]: Listening on Journal Audit Socket.
[[0;32m OK [0m] Listening on [0;1;39mJournal Audit Socket[0m.
[ 4.362897] systemd[1]: Listening on Journal Socket (/dev/log).
[[0;32m OK [0m] Listening on [0;1;39mJournal Socket (/dev/log)[0m.
[ 4.364812] systemd[1]: Listening on Journal Socket.
[[0;32m OK [0m] Listening on [0;1;39mJournal Socket[0m.
[ 4.366481] systemd[1]: Listening on Network Service Netlink Socket.
[[0;32m OK [0m] Listening on [0;1;39mNetwork Service Netlink Socket[0m.
[ 4.368492] systemd[1]: Listening on udev Control Socket.
[[0;32m OK [0m] Listening on [0;1;39mudev Control Socket[0m.
[ 4.370225] systemd[1]: Listening on udev Kernel Socket.
[[0;32m OK [0m] Listening on [0;1;39mudev Kernel Socket[0m.
[ 4.372618] systemd[1]: Mounting Huge Pages File System...
Mounting [0;1;39mHuge Pages File System[0m...
[ 4.375061] systemd[1]: Mounting POSIX Message Queue File System...
Mounting [0;1;39mPOSIX Message Queue File System[0m...
[ 4.377511] systemd[1]: Mounting NFSD configuration filesystem...
Mounting [0;1;39mNFSD configuration filesystem[0m...
[ 4.380065] systemd[1]: Mounting Kernel Debug File System...
Mounting [0;1;39mKernel Debug File System[0m...
[ 4.382521] systemd[1]: Mounting Kernel Trace File System...
Mounting [0;1;39mKernel Trace File System[0m...
[ 4.385870] systemd[1]: Starting Journal Service...
Starting [0;1;39mJournal Service[0m...
[ 4.387468] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS being skipped.
[ 4.389976] systemd[1]: Starting Set the console keyboard layout...
Starting [0;1;39mSet the console keyboard layout[0m...
[ 4.392852] systemd[1]: Starting Create List of Static Device Nodes...
Starting [0;1;39mCreate List of Static Device Nodes[0m...
[ 4.395585] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
Starting [0;1;39mMonitoring of LVM…meventd or progress polling[0m...
[ 4.412546] systemd[1]: Condition check resulted in LXD - agent being skipped.
[ 4.421383] systemd[1]: Starting Load Kernel Module configfs...
Starting [0;1;39mLoad Kernel Module configfs[0m...
[ 4.426660] systemd[1]: Starting Load Kernel Module drm...
Starting [0;1;39mLoad Kernel Module drm[0m...
[ 4.429026] RPC: Registered named UNIX socket transport module.
[ 4.430067] RPC: Registered udp transport module.
[ 4.431048] RPC: Registered tcp transport module.
[ 4.431978] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 4.440998] systemd[1]: Starting Load Kernel Module efi_pstore...
Starting [0;1;39mLoad Kernel Module efi_pstore[0m...
[ 4.447832] systemd[1]: Starting Load Kernel Module fuse...
Starting [0;1;39mLoad Kernel Module fuse[0m...
[ 4.449678] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
[ 4.451485] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[ 4.456020] systemd[1]: Starting Load Kernel Modules...
Starting [0;1;39mLoad Kernel Modules[0m...
[ 4.458430] systemd[1]: Starting Remount Root and Kernel File Systems...
Starting [0;1;39mRemount Root and Kernel File Systems[0m...
[ 4.461401] systemd[1]: Starting Coldplug All udev Devices...
Starting [0;1;39mColdplug All udev Devices[0m...
[ 4.473783] systemd[1]: Started Journal Service.
[[0;32m OK [0m] Started [0;1;39mJournal Service[0m.
[[0;32m OK [0m] Mounted [0;1;39mHuge Pages File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mPOSIX Message Queue File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Debug File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Trace File System[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate List of Static Device Nodes[0m.
[[0;32m OK [0m] Finished [0;1;39mMonitoring of LVM… dmeventd or progress polling[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module configfs[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module drm[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module efi_pstore[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module fuse[0m.
Mounting [0;1;39mFUSE Control File System[0m...
Mounting [0;1;39mKernel Configuration File System[0m...
[[0;32m OK [0m] Mounted [0;1;39mFUSE Control File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Configuration File System[0m.
[ 4.501826] IPMI message handler: version 39.2
[ 4.513518] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[ 4.515305] EXT4-fs (vda1): re-mounted. Opts: (null). Quota mode: none.
[ 4.517770] ipmi device interface
[[0;32m OK [0m] Mounted [0;1;39mNFSD configuration filesystem[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Modules[0m.
[[0;32m OK [0m] Finished [0;1;39mRemount Root and Kernel File Systems[0m.
Activating swap [0;1;39m/swap.img[0m...
[ 4.536608] Adding 4194300k swap on /swap.img. Priority:-2 extents:4 across:5857276k FS
Starting [0;1;39mDevice-Mapper Multipath Device Controller[0m...
[ 4.541855] alua: device handler registered
[ 4.543588] emc: device handler registered
[ 4.546052] rdac: device handler registered
Starting [0;1;39mFlush Journal to Persistent Storage[0m...
Starting [0;1;39mLoad/Save Random Seed[0m...
[ 4.556706] systemd-journald[353]: Received client request to flush runtime journal.
Starting [0;1;39mApply Kernel Variables[0m...
Starting [0;1;39mCreate System Users[0m...
[[0;32m OK [0m] Activated swap [0;1;39m/swap.img[0m.
[[0;32m OK [0m] Finished [0;1;39mSet the console keyboard layout[0m.
[[0;32m OK [0m] Finished [0;1;39mColdplug All udev Devices[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad/Save Random Seed[0m.
[[0;32m OK [0m] Reached target [0;1;39mSwaps[0m.
Starting [0;1;39mHelper to synchronize boot up for ifupdown[0m...
[[0;32m OK [0m] Finished [0;1;39mCreate System Users[0m.
[[0;32m OK [0m] Finished [0;1;39mApply Kernel Variables[0m.
Starting [0;1;39mCreate Static Device Nodes in /dev[0m...
[[0;32m OK [0m] Started [0;1;39mDevice-Mapper Multipath Device Controller[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate Static Device Nodes in /dev[0m.
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Local File Systems[0m.
[[0;32m OK [0m] Reached target [0;1;39mLocal File Systems[0m.
Starting [0;1;39mLoad AppArmor profiles[0m...
Starting [0;1;39mSet console font and keymap[0m...
Starting [0;1;39mCreate final runt…dir for shutdown pivot root[0m...
Starting [0;1;39mTell Plymouth To Write Out Runtime Data[0m...
Starting [0;1;39mSet Up Additional Binary Formats[0m...
Starting [0;1;39mRule-based Manage…for Device Events and Files[0m...
Starting [0;1;39mUncomplicated firewall[0m...
[[0;32m OK [0m] Finished [0;1;39mFlush Journal to Persistent Storage[0m.
[[0;32m OK [0m] Finished [0;1;39mSet console font and keymap[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate final runt…e dir for shutdown pivot root[0m.
[[0;32m OK [0m] Finished [0;1;39mTell Plymouth To Write Out Runtime Data[0m.
[[0;32m OK [0m] Finished [0;1;39mUncomplicated firewall[0m.
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Network[0m.
Mounting [0;1;39mArbitrary Executable File Formats File System[0m...
Starting [0;1;39mCreate Volatile Files and Directories[0m...
[ 4.733882] audit: type=1400 audit(1734356559.544:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=424 comm="apparmor_parser"
[ 4.736391] audit: type=1400 audit(1734356559.544:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=424 comm="apparmor_parser"
[ 4.739088] audit: type=1400 audit(1734356559.548:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lsb_release" pid=423 comm="apparmor_parser"
[[0;32m OK [0m] Mounted [0;1;39mArbitrary Executable File Formats File System[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate Volatile Files and Directories[0m.
Mounting [0;1;39mRPC Pipe File System[0m...
Starting [0;1;39mRPC bind portmap service[0m...
Starting [0;1;39mRecord System Boot/Shutdown in UTMP[0m...
[[0;32m OK [0m] Finished [0;1;39mSet Up Additional Binary Formats[0m.
[[0;32m OK [0m] Mounted [0;1;39mRPC Pipe File System[0m.
[[0;32m OK [0m] Reached target [0;1;39mrpc_pipefs.target[0m.
Starting [0;1;39mpNFS block layout mapping daemon[0m...
Starting [0;1;39mNFSv4 ID-name mapping service[0m...
Starting [0;1;39mNFSv4 Client Tracking Daemon[0m...
[[0;32m OK [0m] Reached target [0;1;39mNFS client services[0m.
[[0;32m OK [0m] Started [0;1;39mRPC bind portmap service[0m.
[[0;32m OK [0m] Started [0;1;39mpNFS block layout mapping daemon[0m.
[[0;32m OK [0m] Started [0;1;39mNFSv4 ID-name mapping service[0m.
[[0;32m OK [0m] Started [0;1;39mNFSv4 Client Tracking Daemon[0m.
[[0;32m OK [0m] Reached target [0;1;39mRPC Port Mapper[0m.
[[0;32m OK [0m] Finished [0;1;39mRecord System Boot/Shutdown in UTMP[0m.
[ 4.785125] audit: type=1400 audit(1734356559.596:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=427 comm="apparmor_parser"
[ 4.788230] audit: type=1400 audit(1734356559.596:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=427 comm="apparmor_parser"
[ 4.791125] audit: type=1400 audit(1734356559.596:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=427 comm="apparmor_parser"
[ 4.794974] audit: type=1400 audit(1734356559.596:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/{,usr/}sbin/dhclient" pid=427 comm="apparmor_parser"
[ 4.797962] audit: type=1400 audit(1734356559.600:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=428 comm="apparmor_parser"
[ 4.801830] audit: type=1400 audit(1734356559.600:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=428 comm="apparmor_parser"
[[0;32m OK [0m] Started [0;1;39mRule-based Manager for Device Events and Files[0m.
[[0;32m OK [0m] Started [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[[0;32m OK [0m] Reached target [0;1;39mLocal Encrypted Volumes[0m.
Starting [0;1;39mNetwork Configuration[0m...
[[0;32m OK [0m] Finished [0;1;39mLoad AppArmor profiles[0m.
[[0;32m OK [0m] Reached target [0;1;39mSystem Initialization[0m.
[[0;32m OK [0m] Started [0;1;39mTrigger to poll fo…y enabled on GCP LTS non-pro)[0m.
[[0;32m OK [0m] Started [0;1;39mDaily Cleanup of Temporary Directories[0m.
[[0;32m OK [0m] Started [0;1;39mUbuntu Advantage Timer for running repeated jobs[0m.
[[0;32m OK [0m] Reached target [0;1;39mPath Units[0m.
[[0;32m OK [0m] Listening on [0;1;39mD-Bus System Message Bus Socket[0m.
[[0;32m OK [0m] Listening on [0;1;39mOpen-iSCSI iscsid Socket[0m.
[[0;32m OK [0m] Listening on [0;1;39mUUID daemon activation socket[0m.
[[0;32m OK [0m] Reached target [0;1;39mSocket Units[0m.
[[0;32m OK [0m] Reached target [0;1;39mBasic System[0m.
[[0;32m OK [0m] Started [0;1;39mD-Bus System Message Bus[0m.
Starting [0;1;39mNetwork Manager[0m...
[[0;32m OK [0m] Started [0;1;39mSave initial kernel messages after boot[0m.
Starting [0;1;39mRemove Stale Onli…t4 Metadata Check Snapshots[0m...
Starting [0;1;39mRecord successful boot for GRUB[0m...
[[0;32m OK [0m] Started [0;1;39mirqbalance daemon[0m.
Starting [0;1;39mKernel crash dump capture service[0m...
Starting [0;1;39mLSB: Execute the …-e command to reboot system[0m...
Starting [0;1;39mDispatcher daemon for systemd-networkd[0m...
Starting [0;1;39mAuthorization Manager[0m...
Starting [0;1;39mSystem Logging Service[0m...
Starting [0;1;39mUser Login Management[0m...
Starting [0;1;39mDisk Manager[0m...
Starting [0;1;39mWPA supplicant[0m...
[[0;32m OK [0m] Started [0;1;39mNetwork Configuration[0m.
[[0;32m OK [0m] Finished [0;1;39mRemove Stale Onli…ext4 Metadata Check Snapshots[0m.
[[0;32m OK [0m] Started [0;1;39mSystem Logging Service[0m.
[[0;32m OK [0m] Started [0;1;39mNetwork Manager[0m.
[[0;32m OK [0m] Started [0;1;39mAuthorization Manager[0m.
[[0;32m OK [0m] Started [0;1;39mWPA supplicant[0m.
[[0;32m OK [0m] Found device [0;1;39m/dev/ttyS0[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Execute the k…c -e command to reboot system[0m.
[[0;32m OK [0m] Finished [0;1;39mRecord successful boot for GRUB[0m.
[[0;32m OK [0m] Started [0;1;39mUser Login Management[0m.
[ 5.084027] kdump-tools[470]: Starting kdump-tools:
[ 5.084905] kdump-tools[510]: * Creating symlink /var/lib/kdump/vmlinuz
Starting [0;1;39mModem Manager[0m...
[ 5.098622] kdump-tools[510]: * Creating symlink /var/lib/kdump/initrd.img
Starting [0;1;39mNetwork Manager Wait Online[0m...
Starting [0;1;39mGRUB failed boot detection[0m...
Starting [0;1;39mHostname Service[0m...
Starting [0;1;39mWait for Network to be Configured[0m...
Starting [0;1;39mNetwork Name Resolution[0m...
[[0;32m OK [0m] Started [0;1;39mDisk Manager[0m.
[[0;32m OK [0m] Finished [0;1;39mGRUB failed boot detection[0m.
[[0;32m OK [0m] Started [0;1;39mHostname Service[0m.
Starting [0;1;39mNetwork Manager Script Dispatcher Service[0m...
[[0;32m OK [0m] Started [0;1;39mNetwork Manager Script Dispatcher Service[0m.
[[0;32m OK [0m] Finished [0;1;39mNetwork Manager Wait Online[0m.
[[0;32m OK [0m] Started [0;1;39mNetwork Name Resolution[0m.
[[0;32m OK [0m] Reached target [0;1;39mHost and Network Name Lookups[0m.
[[0;32m OK [0m] Started [0;1;39mModem Manager[0m.
[[0;32m OK [0m] Started [0;1;39mDispatcher daemon for systemd-networkd[0m.
[[0;32m OK [0m] Listening on [0;1;39mLoad/Save RF …itch Status /dev/rfkill Watch[0m.
[[0;32m OK [0m] Found device [0;1;39mVirtio network device[0m.
[[0;32m OK [0m] Started [0;1;39mifup for eth0[0m.
[[0;32m OK [0m] Finished [0;1;39mHelper to synchronize boot up for ifupdown[0m.
Starting [0;1;39mRaise network interfaces[0m...
[ 6.188034] IPv6: eth0: IPv6 duplicate address fd33:3981:3213:f020:0:5254:82:c0a8 used by 52:54:00:6a:a7:e6 detected!
[ 6.471763] kdump-tools[510]: * loaded kdump kernel
[[0;32m OK [0m] Finished [0;1;39mKernel crash dump capture service[0m.
[[0m[0;31m* [0m] (1 of 2) A start job is running for…k to be Configured (4s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…k to be Configured (5s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for…k to be Configured (5s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for… network interfaces (6s / 5min 1s)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for… network interfaces (6s / 5min 1s)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for… network interfaces (7s / 5min 1s)
M
[K[ [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for…k to be Configured (7s / no limit)
M
[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mRaise network interfaces[0m.
[KSee 'systemctl status networking.service' for details.
[[0;32m OK [0m] Reached target [0;1;39mNetwork[0m.
Starting [0;1;39mchrony, an NTP client/server[0m...
[[0;32m OK [0m] Started [0;1;39mUnattended Upgrades Shutdown[0m.
[[0;32m OK [0m] Started [0;1;39mchrony, an NTP client/server[0m.
[[0;32m OK [0m] Reached target [0;1;39mSystem Time Synchronized[0m.
[[0;32m OK [0m] Started [0;1;39mDaily apt download activities[0m.
[[0;32m OK [0m] Started [0;1;39mDaily apt upgrade and clean activities[0m.
[[0;32m OK [0m] Started [0;1;39mDaily dpkg database backup timer[0m.
[[0;32m OK [0m] Started [0;1;39mPeriodic ext4 Onli…ata Check for All Filesystems[0m.
[[0;32m OK [0m] Started [0;1;39mDiscard unused blocks once a week[0m.
[[0;32m OK [0m] Started [0;1;39mRefresh fwupd metadata regularly[0m.
[[0;32m OK [0m] Started [0;1;39mDaily rotation of log files[0m.
[[0;32m OK [0m] Started [0;1;39mDaily man-db regeneration[0m.
[[0;32m OK [0m] Started [0;1;39mMessage of the Day[0m.
Starting [0;1;39mMUNGE authentication service[0m...
[[0;32m OK [0m] Started [0;1;39mMUNGE authentication service[0m.
[ [0;31m*[0m] A start job is running for Wait for… to be Configured (10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (11s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (11s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (12s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (12s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (13s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (14s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (14s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (15s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (15s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (16s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (16s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (18s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (18s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (19s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (19s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (20s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (21s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (21s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (22s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (22s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (26s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (26s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (27s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (29s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (29s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (30s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (30s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (31s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (33s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (33s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (34s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (36s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (36s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (37s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (37s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (40s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (40s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (41s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (41s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (43s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (43s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (44s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (44s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (47s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (47s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (48s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (48s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (50s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (50s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (51s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (51s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (54s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (54s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (55s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (55s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (57s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (57s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (58s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (58s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…to be Configured (2min / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (2min / no limit)
M
[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mWait for Network to be Configured[0m.
[KSee 'systemctl status systemd-networkd-wait-online.service' for details.
[[0;32m OK [0m] Reached target [0;1;39mNetwork is Online[0m.
[[0;32m OK [0m] Started [0;1;39mDownload data for …ailed at package install time[0m.
[[0;32m OK [0m] Started [0;1;39mCheck to see wheth…w version of Ubuntu available[0m.
[[0;32m OK [0m] Reached target [0;1;39mTimer Units[0m.
Starting [0;1;39mNFS Mount Daemon[0m...
Starting [0;1;39mSamba NMB Daemon[0m...
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Remote File Systems[0m.
Mounting [0;1;39m/autotest[0m...
Mounting [0;1;39m/scratch[0m...
[[0;32m OK [0m] Finished [0;1;39mAvailability of block devices[0m.
[ 125.365930] FS-Cache: Loaded
Starting [0;1;39mPostfix Mail Transport Agent (instance -)[0m...
[ 125.386812] FS-Cache: Netfs 'nfs' registered for caching
Starting [0;1;39mNFS status monitor for NFSv2/3 locking.[0m...
Starting [0;1;39mOpenBSD Secure Shell server[0m...
[[0;32m OK [0m] Started [0;1;39mNFS Mount Daemon[0m.
[[0;32m OK [0m] Started [0;1;39mSamba NMB Daemon[0m.
[[0;32m OK [0m] Started [0;1;39mNFS status monitor for NFSv2/3 locking.[0m.
Starting [0;1;39mNFS server and services[0m...
Starting [0;1;39mSamba SMB Daemon[0m...
[[0;32m OK [0m] Started [0;1;39mOpenBSD Secure Shell server[0m.
[ 125.544114] NFS: Registering the id_resolver key type
[ 125.545669] Key type id_resolver registered
[ 125.546886] Key type id_legacy registered
[ 125.557379] FS-Cache: Duplicate cookie detected
[ 125.558308] FS-Cache: O-cookie c=00000003 [p=00000002 fl=222 nc=0 na=1]
[ 125.559501] FS-Cache: O-cookie d=000000001ced26fb{NFS.server} n=000000002324d8a7
[ 125.560824] FS-Cache: O-key=[16] '0400000002000000020008010af020cc'
[ 125.561942] FS-Cache: N-cookie c=00000004 [p=00000002 fl=2 nc=0 na=1]
[ 125.563081] FS-Cache: N-cookie d=000000001ced26fb{NFS.server} n=00000000605bfe6e
[ 125.564380] FS-Cache: N-key=[16] '0400000002000000020008010af020cc'
[[0;32m OK [0m] Mounted [0;1;39m/autotest[0m.
[[0;32m OK [0m] Mounted [0;1;39m/scratch[0m.
[[0;32m OK [0m] Reached target [0;1;39mRemote File Systems[0m.
Starting [0;1;39mLSB: automatic crash report generation[0m...[ 125.665744] NFSD: Using nfsdcld client tracking operations.
[ 125.666796] NFSD: no clients to reclaim, skipping NFSv4 grace period (net f0000000)
Starting [0;1;39mAutomounts filesystems on demand[0m...
[[0;32m OK [0m] Started [0;1;39mRegular background program processing daemon[0m.
Starting [0;1;39mLSB: Load kernel image with kexec[0m...
Starting [0;1;39mLSB: Start/Stop t…mote power manager) daemon.[0m...
Starting [0;1;39mPermit User Sessions[0m...
Starting [0;1;39mLSB: Start vmtouch daemon[0m...
Starting [0;1;39mLSB: Starts or stops the xinetd daemon.[0m...
[[0;32m OK [0m] Finished [0;1;39mPermit User Sessions[0m.
Starting [0;1;39mHold until boot process finishes up[0m...
Starting [0;1;39mTerminate Plymouth Boot Screen[0m...
[[0;32m OK [0m] Finished [0;1;39mHold until boot process finishes up[0m.
[[0;32m OK [0m] Started [0;1;39mSerial Getty on ttyS0[0m.
Starting [0;1;39mSet console scheme[0m...
[[0;32m OK [0m] Finished [0;1;39mTerminate Plymouth Boot Screen[0m.
[[0;32m OK [0m] Finished [0;1;39mSet console scheme[0m.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/getty[0m.
[[0;32m OK [0m] Started [0;1;39mGetty on tty1[0m.
[[0;32m OK [0m] Reached target [0;1;39mLogin Prompts[0m.
[[0;32m OK [0m] Started [0;1;39mSamba SMB Daemon[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Load kernel image with kexec[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Start vmtouch daemon[0m.
[[0;32m OK [0m] Finished [0;1;39mNFS server and services[0m.
Starting [0;1;39mNotify NFS peers of a restart[0m...
[[0;32m OK [0m] Started [0;1;39mLSB: Start/Stop th…remote power manager) daemon.[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: automatic crash report generation[0m.
[[0;32m OK [0m] Started [0;1;39mNotify NFS peers of a restart[0m.
[[0;32m OK [0m] Started [0;1;39mAutomounts filesystems on demand[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Starts or stops the xinetd daemon.[0m.
[[0;32m OK [0m] Started [0;1;39mPostfix Mail Transport Agent (instance -)[0m.
Starting [0;1;39mPostfix Mail Transport Agent[0m...
[[0;32m OK [0m] Finished [0;1;39mPostfix Mail Transport Agent[0m.
[[0;32m OK [0m] Reached target [0;1;39mMulti-User System[0m.
[[0;32m OK [0m] Reached target [0;1;39mGraphical Interface[0m.
Starting [0;1;39mRecord Runlevel Change in UTMP[0m...
[[0;32m OK [0m] Finished [0;1;39mRecord Runlevel Change in UTMP[0m.
Ubuntu 22.04 LTS trevis-106vm14.trevis.whamcloud.com ttyS0
trevis-106vm14 login: [ 170.343542] Autotest: Nodes configured in 3 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 170.944724] Autotest: Node provisioning/configuration took 83 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 171.761013] Autotest: Starting test framework on controller trevis-106vm14 (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 173.655533] Autotest: Test framework started (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 174.181723] Autotest: Test framework pid=2775 (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 174.211420] libcfs: loading out-of-tree module taints kernel.
[ 174.212611] libcfs: module verification failed: signature and/or required key missing - tainting kernel
[ 174.215601] Key type ._llcrypt registered
[ 174.216363] Key type .llcrypt registered
[ 174.232382] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[ 174.824431] Autotest: Test running for 0 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 177.869549] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 177.879867] Lustre: DEBUG MARKER: /usr/sbin/lctl lustre_build_version 2>/dev/null
[ 181.053105] Lustre: DEBUG MARKER: cat /etc/system-release
[ 181.063733] Lustre: DEBUG MARKER: test -r /etc/os-release
[ 181.073711] Lustre: DEBUG MARKER: cat /etc/os-release
[ 181.324043] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 181.324043] if [ $running -ne 0 ] ; then
[ 181.324043] echo Stopping client $(hostname) /mnt/lustre opts:-f;
[ 181.324043] lsof /mnt/lustre || need_kill=no;
[ 181.324043] if [ x-f != x -a x$need_kill != xno ]; then
[ 181.324043] pids=$(lsof -t /mnt/lustre | sort -u);
[ 181.324043]
[ 181.929008] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 181.929008] if [ $running -ne 0 ] ; then
[ 181.929008] echo Stopping client $(hostname) /mnt/lustre2 opts:-f;
[ 181.929008] lsof /mnt/lustre2 || need_kill=no;
[ 181.929008] if [ x-f != x -a x$need_kill != xno ]; then
[ 181.929008] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 197.284679] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 197.444819] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 197.446024] LNet: Accept all, port 7988
[ 198.981529] Key type lgssc registered
[ 199.134393] Lustre: Echo OBD driver; http://www.lustre.org/
[ 341.450992] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 341.450992] mpts=$(mount | grep -c /mnt/lustre' ');
[ 341.450992] if [ $running -ne $mpts ]; then
[ 341.450992] echo $(hostname) env are INSANE!;
[ 341.450992] exit 1;
[ 341.450992] fi
[ 342.085462] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 342.085462] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 342.085462] if [ $running -ne $mpts ]; then
[ 342.085462] echo $(hostname) env are INSANE!;
[ 342.085462] exit 1;
[ 342.085462] fi
[ 435.981608] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 435.990951] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 437.028583] Lustre: Mounted lustre-client
[ 437.226019] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 437.846549] Lustre: DEBUG MARKER:
[ 437.846549] running=$(mount | grep -c /mnt/lustre' ');
[ 437.846549] rc=0;
[ 437.846549] if [ $running -eq 0 ] ; then
[ 437.846549] mkdir -p /mnt/lustre;
[ 437.846549] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 437.846549] rc=$?;
[ 437.846549] fi;
[ 437.846549] exit $rc
[ 439.695319] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 444.496827] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 444.822025] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 445.185610] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 445.990148] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 446.015306] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 446.039149] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 446.401899] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 447.414822] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 448.427925] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 449.440587] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 450.453465] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 451.466644] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 452.479703] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 453.492895] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 453.876945] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[ 453.886818] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[ 461.797025] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/li
[ 462.694327] Lustre: lustre-OST0000-osc-ffff896185c07000: disconnect after 20s idle
[ 463.123762] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing check_logdir \/autotest\/autotest-1\/2024-12-16\/lustre-reviews_review-ldiskfs-ubuntu_109661_43_5ffd44ab-3e0f-49e9-a7ff-7a720f91c9d8\/
[ 463.457376] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing check_logdir /autotest/autotest-1/2024-12-16/lustre-reviews_review-ldiskfs-ubuntu_109661_43_5ffd44ab-3e0f-49e9-a7ff-7a720f91c9d8/
[ 464.774970] Lustre: DEBUG MARKER: uname -n
[ 466.301302] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/li
[ 467.653691] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing yml_node
[ 468.026571] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing yml_node
[ 468.351194] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 469.647258] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 469.920523] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Client: 2.16.50.134
[ 470.250413] Lustre: DEBUG MARKER: Client: 2.16.50.134
[ 470.982025] Lustre: DEBUG MARKER: /usr/sbin/lctl mark MDS: 2.16.50.134
[ 471.306911] Lustre: DEBUG MARKER: MDS: 2.16.50.134
[ 472.062929] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OSS: 2.16.50.134
[ 472.391605] Lustre: DEBUG MARKER: OSS: 2.16.50.134
[ 472.779253] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: lnet-selftest ============----- Mon Dec 16 13:50:26 UTC 2024
[ 473.109868] Lustre: DEBUG MARKER: -----============= acceptance-small: lnet-selftest ============----- Mon Dec 16 13:50:26 UTC 2024
[ 474.862591] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 475.995162] Autotest: Test running for 5 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 478.205258] Lustre: DEBUG MARKER: cat /etc/system-release
[ 478.216654] Lustre: DEBUG MARKER: test -r /etc/os-release
[ 478.227391] Lustre: DEBUG MARKER: cat /etc/os-release
[ 479.351620] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests:
[ 479.685754] Lustre: DEBUG MARKER: excepting tests:
[ 486.169253] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 486.959874] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 486.959874] if [ $running -ne 0 ] ; then
[ 486.959874] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 486.959874] lsof /mnt/lustre || need_kill=no;
[ 486.959874] if [ x != x -a x$need_kill != xno ]; then
[ 486.959874] pids=$(lsof -t /mnt/lustre | sort -u);
[ 486.959874] if
[ 487.819710] Lustre: Unmounted lustre-client
[ 488.448357] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == lnet-selftest test smoke: lst regression test ========= 13:50:42 \(1734357042\)
[ 488.772829] Lustre: DEBUG MARKER: == lnet-selftest test smoke: lst regression test ========= 13:50:42 (1734357042)
[ 489.137015] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 490.419679] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing lst_cleanup
[ 490.757421] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing lst_cleanup
[ 492.313834] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 493.569883] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing lst_setup
[ 493.900845] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing lst_setup
[ 495.410055] LNet: 14176:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 495.514769] LNet: 14176:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 495.618772] LNet: 14176:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 495.722767] LNet: 14176:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 495.826782] LNet: 14176:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 495.930763] LNet: 14176:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 496.034792] LNet: 14176:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 497.221741] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 497.442027] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 497.664024] LNet: 9:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 497.897884] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 498.118175] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 498.337914] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 498.568141] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 498.802240] LNet: 9:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 499.026230] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 499.246244] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 499.465722] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 499.686126] LNet: 97:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[ 777.353761] Autotest: Test running for 10 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 802.361063] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 803.901140] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing lst_cleanup
[ 804.255819] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing lst_cleanup
[ 804.580282] LNet: 14656:0:(timer.c:220:stt_shutdown()) waiting for 1 threads to terminate
[ 807.076066] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == lnet-selftest test complete, duration 334 sec ========= 13:56:01 \(1734357361\)
[ 807.408458] Lustre: DEBUG MARKER: == lnet-selftest test complete, duration 334 sec ========= 13:56:01 (1734357361)
[ 807.722562] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 808.346927] Lustre: DEBUG MARKER:
[ 808.346927] running=$(mount | grep -c /mnt/lustre' ');
[ 808.346927] rc=0;
[ 808.346927] if [ $running -eq 0 ] ; then
[ 808.346927] mkdir -p /mnt/lustre;
[ 808.346927] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 808.346927] rc=$?;
[ 808.346927] fi;
[ 808.346927] exit $rc
[ 808.718218] Lustre: Mounted lustre-client
[ 809.013957] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 811.628551] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === lnet-selftest: start cleanup 13:56:05 \(1734357365\) ===
[ 811.955669] Lustre: DEBUG MARKER: === lnet-selftest: start cleanup 13:56:05 (1734357365) ===
[ 812.781689] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === lnet-selftest: finish cleanup 13:56:06 \(1734357366\) ===
[ 813.085678] Lustre: DEBUG MARKER: === lnet-selftest: finish cleanup 13:56:06 (1734357366) ===
[ 813.479628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: sanity ============----- Mon Dec 16 13:56:07 UTC 2024
[ 813.806815] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity ============----- Mon Dec 16 13:56:07 UTC 2024
[ 815.516419] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 818.731266] Lustre: DEBUG MARKER: cat /etc/system-release
[ 818.742487] Lustre: DEBUG MARKER: test -r /etc/os-release
[ 818.752893] Lustre: DEBUG MARKER: cat /etc/os-release
[ 819.829459] Lustre: DEBUG MARKER: /usr/sbin/lctl mark - need mds1 \< v2_14_55-100-g8a84c7f9c7 for LU-14927, skip 0f
[ 820.171981] Lustre: DEBUG MARKER: - need mds1 < v2_14_55-100-g8a84c7f9c7 for LU-14927, skip 0f
[ 820.549146] Lustre: DEBUG MARKER: /usr/sbin/lctl mark - need mds1 \<= 2.14.55-100-g8a84c7f9c7 for LU-14927, skip 0f
[ 820.879197] Lustre: DEBUG MARKER: - need mds1 <= 2.14.55-100-g8a84c7f9c7 for LU-14927, skip 0f
[ 821.253561] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests: 0f 0f 0f 42a 42c 42b 118c 118d 407 119i 817
[ 821.585201] Lustre: DEBUG MARKER: excepting tests: 0f 0f 0f 42a 42c 42b 118c 118d 407 119i 817
[ 821.953934] Lustre: DEBUG MARKER: /usr/sbin/lctl mark skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842
[ 822.278840] Lustre: DEBUG MARKER: skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842
[ 822.653706] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: start setup 13:56:16 \(1734357376\) ===
[ 822.979866] Lustre: DEBUG MARKER: === sanity: start setup 13:56:16 (1734357376) ===
[ 823.312574] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 824.438077] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing check_config_client \/mnt\/lustre
[ 824.813724] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing check_config_client /mnt/lustre
[ 829.279294] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 829.279294] mpts=$(mount | grep -c /mnt/lustre' ');
[ 829.279294] if [ $running -ne $mpts ]; then
[ 829.279294] echo $(hostname) env are INSANE!;
[ 829.279294] exit 1;
[ 829.279294] fi
[ 829.920790] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 829.920790] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 829.920790] if [ $running -ne $mpts ]; then
[ 829.920790] echo $(hostname) env are INSANE!;
[ 829.920790] exit 1;
[ 829.920790] fi
[ 838.387701] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 838.712744] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 839.080294] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 839.887470] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 844.407915] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: finish setup 13:56:38 \(1734357398\) ===
[ 844.740358] Lustre: DEBUG MARKER: === sanity: finish setup 13:56:38 (1734357398) ===
[ 845.476901] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0a: touch\; rm ============================= 13:56:39 \(1734357399\)
[ 845.804738] Lustre: DEBUG MARKER: == sanity test 0a: touch; rm ============================= 13:56:39 (1734357399)
[ 847.945298] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0b: chmod 0755 \/mnt\/lustre ======================================================================================= 13:56:42 \(1734357402\)
[ 848.274794] Lustre: DEBUG MARKER: == sanity test 0b: chmod 0755 /mnt/lustre ======================================================================================= 13:56:42 (1734357402)
[ 849.865031] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0c: check import proc ===================== 13:56:44 \(1734357404\)
[ 850.197301] Lustre: DEBUG MARKER: == sanity test 0c: check import proc ===================== 13:56:44 (1734357404)
[ 851.802001] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0d: check export proc ======================================================================================= 13:56:45 \(1734357405\)
[ 852.130686] Lustre: DEBUG MARKER: == sanity test 0d: check export proc ======================================================================================= 13:56:45 (1734357405)
[ 853.720172] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 855.214230] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0e: Enable DNE MDT balancing for mkdir in the ROOT ========================================================== 13:56:49 \(1734357409\)
[ 855.542932] Lustre: DEBUG MARKER: == sanity test 0e: Enable DNE MDT balancing for mkdir in the ROOT ========================================================== 13:56:49 (1734357409)
[ 855.922166] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_0e We need at least 2 MDTs for this test
[ 856.238231] Lustre: DEBUG MARKER: SKIP: sanity test_0e We need at least 2 MDTs for this test
[ 856.629313] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 1: mkdir\; remkdir\; rmdir ================== 13:56:50 \(1734357410\)
[ 856.956733] Lustre: DEBUG MARKER: == sanity test 1: mkdir; remkdir; rmdir ================== 13:56:50 (1734357410)
[ 858.568974] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 2: mkdir\; touch\; rmdir\; check file ======== 13:56:52 \(1734357412\)
[ 858.900667] Lustre: DEBUG MARKER: == sanity test 2: mkdir; touch; rmdir; check file ======== 13:56:52 (1734357412)
[ 860.620232] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 3: mkdir\; touch\; rmdir\; check dir ========= 13:56:54 \(1734357414\)
[ 860.954581] Lustre: DEBUG MARKER: == sanity test 3: mkdir; touch; rmdir; check dir ========= 13:56:54 (1734357414)
[ 862.697906] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 4: mkdir\; touch dir\/file\; rmdir\; checkdir \(expect error\) ========================================================== 13:56:56 \(1734357416\)
[ 863.035565] Lustre: DEBUG MARKER: == sanity test 4: mkdir; touch dir/file; rmdir; checkdir (expect error) ========================================================== 13:56:56 (1734357416)
[ 864.934589] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 5: mkdir ...\/d5 ...\/d5\/d2\; chmod ...\/d5\/d2 ========================================================== 13:56:59 \(1734357419\)
[ 865.131951] Lustre: lustre-OST0004-osc-ffff89618e2ab000: disconnect after 20s idle
[ 865.133353] Lustre: Skipped 6 previous similar messages
[ 865.271574] Lustre: DEBUG MARKER: == sanity test 5: mkdir .../d5 .../d5/d2; chmod .../d5/d2 ========================================================== 13:56:59 (1734357419)
[ 866.915445] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6a: touch f6a\; chmod f6a\; runas -u 500 -g 500 chmod f6a \(should return error\) ============================================================ 13:57:01 \(1734357421\)
[ 867.245998] Lustre: DEBUG MARKER: == sanity test 6a: touch f6a; chmod f6a; runas -u 500 -g 500 chmod f6a (should return error) ============================================================ 13:57:01 (1734357421)
[ 868.922550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6c: touch f6c\; chown f6c\; runas -u 500 -g 500 chown f6c \(should return error\) ============================================================ 13:57:03 \(1734357423\)
[ 869.256247] Lustre: DEBUG MARKER: == sanity test 6c: touch f6c; chown f6c; runas -u 500 -g 500 chown f6c (should return error) ============================================================ 13:57:03 (1734357423)
[ 870.865923] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6e: touch+chgrp \; runas -u 500 -g 500 chgrp \(should return error\) ========================================================== 13:57:05 \(1734357425\)
[ 871.200028] Lustre: DEBUG MARKER: == sanity test 6e: touch+chgrp ; runas -u 500 -g 500 chgrp (should return error) ========================================================== 13:57:05 (1734357425)
[ 872.814911] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6g: verify new dir in sgid dir inherits group ========================================================== 13:57:06 \(1734357426\)
[ 873.151307] Lustre: DEBUG MARKER: == sanity test 6g: verify new dir in sgid dir inherits group ========================================================== 13:57:06 (1734357426)
[ 874.766940] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6h: runas -u 500 -g 500 chown RUNAS_ID.0 ...\/ \(should return error\) ========================================================== 13:57:08 \(1734357428\)
[ 875.096278] Lustre: DEBUG MARKER: == sanity test 6h: runas -u 500 -g 500 chown RUNAS_ID.0 .../ (should return error) ========================================================== 13:57:08 (1734357428)
[ 876.710995] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 7a: mkdir ...\/d7\; mcreate ...\/d7\/f\; chmod ...\/d7\/f ============================================================== 13:57:10 \(1734357430\)
[ 877.036331] Lustre: DEBUG MARKER: == sanity test 7a: mkdir .../d7; mcreate .../d7/f; chmod .../d7/f ============================================================== 13:57:10 (1734357430)
[ 878.637028] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 7b: mkdir ...\/d7\; mcreate d7\/f2\; echo foo \> d7\/f2 =============================================================== 13:57:12 \(1734357432\)
[ 878.975638] Lustre: DEBUG MARKER: == sanity test 7b: mkdir .../d7; mcreate d7/f2; echo foo > d7/f2 =============================================================== 13:57:12 (1734357432)
[ 880.607432] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 8: mkdir ...\/d8\; touch ...\/d8\/f\; chmod ...\/d8\/f ================================================================= 13:57:14 \(1734357434\)
[ 880.941291] Lustre: DEBUG MARKER: == sanity test 8: mkdir .../d8; touch .../d8/f; chmod .../d8/f ================================================================= 13:57:14 (1734357434)
[ 882.552912] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 9: mkdir ...\/d9 ...\/d9\/d2 ...\/d9\/d2\/d3 ========================================================================== 13:57:16 \(1734357436\)
[ 882.888880] Lustre: DEBUG MARKER: == sanity test 9: mkdir .../d9 .../d9/d2 .../d9/d2/d3 ========================================================================== 13:57:16 (1734357436)
[ 884.513962] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 10: mkdir ...\/d10 ...\/d10\/d2\; touch ...\/d10\/d2\/f ================================================================ 13:57:18 \(1734357438\)
[ 884.845267] Lustre: DEBUG MARKER: == sanity test 10: mkdir .../d10 .../d10/d2; touch .../d10/d2/f ================================================================ 13:57:18 (1734357438)
[ 886.484624] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 11: mkdir ...\/d11 d11\/d2\; chmod ...\/d11\/d2 ====================================================================== 13:57:20 \(1734357440\)
[ 886.817892] Lustre: DEBUG MARKER: == sanity test 11: mkdir .../d11 d11/d2; chmod .../d11/d2 ====================================================================== 13:57:20 (1734357440)
[ 888.436897] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 12: touch ...\/d12\/f\; chmod ...\/d12\/f ...\/d12\/f ================================================================== 13:57:22 \(1734357442\)
[ 888.768321] Lustre: DEBUG MARKER: == sanity test 12: touch .../d12/f; chmod .../d12/f .../d12/f ================================================================== 13:57:22 (1734357442)
[ 890.382776] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 13: creat ...\/d13\/f\; dd ...\/d13\/f\; \> ...\/d13\/f ================================================================== 13:57:24 \(1734357444\)
[ 890.715130] Lustre: DEBUG MARKER: == sanity test 13: creat .../d13/f; dd .../d13/f; > .../d13/f ================================================================== 13:57:24 (1734357444)
[ 892.340948] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 14: touch ...\/d14\/f\; rm ...\/d14\/f\; rm ...\/d14\/f ================================================================= 13:57:26 \(1734357446\)
[ 892.671301] Lustre: DEBUG MARKER: == sanity test 14: touch .../d14/f; rm .../d14/f; rm .../d14/f ================================================================= 13:57:26 (1734357446)
[ 894.290442] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 15: touch ...\/d15\/f\; mv ...\/d15\/f ...\/d15\/f2 ==================================================================== 13:57:28 \(1734357448\)
[ 894.621661] Lustre: DEBUG MARKER: == sanity test 15: touch .../d15/f; mv .../d15/f .../d15/f2 ==================================================================== 13:57:28 (1734357448)
[ 896.260210] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 16: touch ...\/d16\/f\; rm -rf ...\/d16\/f ===== 13:57:30 \(1734357450\)
[ 896.590958] Lustre: DEBUG MARKER: == sanity test 16: touch .../d16/f; rm -rf .../d16/f ===== 13:57:30 (1734357450)
[ 898.203248] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17a: symlinks: create, remove \(real\) ====== 13:57:32 \(1734357452\)
[ 898.534679] Lustre: DEBUG MARKER: == sanity test 17a: symlinks: create, remove (real) ====== 13:57:32 (1734357452)
[ 900.165057] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17b: symlinks: create, remove \(dangling\) == 13:57:34 \(1734357454\)
[ 900.492747] Lustre: DEBUG MARKER: == sanity test 17b: symlinks: create, remove (dangling) == 13:57:34 (1734357454)
[ 902.091486] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17c: symlinks: open dangling \(should return error\) ========================================================== 13:57:36 \(1734357456\)
[ 902.424787] Lustre: DEBUG MARKER: == sanity test 17c: symlinks: open dangling (should return error) ========================================================== 13:57:36 (1734357456)
[ 904.032771] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17d: symlinks: create dangling ============ 13:57:38 \(1734357458\)
[ 904.357938] Lustre: DEBUG MARKER: == sanity test 17d: symlinks: create dangling ============ 13:57:38 (1734357458)
[ 905.978226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17e: symlinks: create recursive symlink \(should return error\) ========================================================== 13:57:40 \(1734357460\)
[ 906.308786] Lustre: DEBUG MARKER: == sanity test 17e: symlinks: create recursive symlink (should return error) ========================================================== 13:57:40 (1734357460)
[ 907.937120] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17f: symlinks: long and very long symlink name ========================================================== 13:57:42 \(1734357462\)
[ 908.269841] Lustre: DEBUG MARKER: == sanity test 17f: symlinks: long and very long symlink name ========================================================== 13:57:42 (1734357462)
[ 909.892761] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17g: symlinks: really long symlink name and inode boundaries ========================================================== 13:57:44 \(1734357464\)
[ 910.224450] Lustre: DEBUG MARKER: == sanity test 17g: symlinks: really long symlink name and inode boundaries ========================================================== 13:57:44 (1734357464)
[ 911.968470] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17h: create objects: lov_free_memmd\(\) doesn\'t lbug ========================================================== 13:57:46 \(1734357466\)
[ 912.298259] Lustre: DEBUG MARKER: == sanity test 17h: create objects: lov_free_memmd() doesn't lbug ========================================================== 13:57:46 (1734357466)
[ 914.341253] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17i: don\'t panic on short symlink \(should return error\) ========================================================== 13:57:48 \(1734357468\)
[ 914.680981] Lustre: DEBUG MARKER: == sanity test 17i: don't panic on short symlink (should return error) ========================================================== 13:57:48 (1734357468)
[ 915.245132] LustreError: 34437:0:(symlink.c:104:ll_readlink_internal()) lustre: inode [0x200000401:0x4f:0x0]: symlink length 33 not expected 35
[ 916.727255] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17k: symlinks: rsync with xattrs enabled == 13:57:50 \(1734357470\)
[ 917.056288] Lustre: DEBUG MARKER: == sanity test 17k: symlinks: rsync with xattrs enabled == 13:57:50 (1734357470)
[ 918.817832] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17l: Ensure lgetxattr\'s returned xattr size is consistent ========================================================== 13:57:52 \(1734357472\)
[ 919.157089] Lustre: DEBUG MARKER: == sanity test 17l: Ensure lgetxattr's returned xattr size is consistent ========================================================== 13:57:52 (1734357472)
[ 920.846248] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17m: run e2fsck against MDT which contains short\/long symlink ========================================================== 13:57:54 \(1734357474\)
[ 921.178933] Lustre: DEBUG MARKER: == sanity test 17m: run e2fsck against MDT which contains short/long symlink ========================================================== 13:57:54 (1734357474)
[ 936.178801] LustreError: lustre-MDT0000-mdc-ffff89618e2ab000: operation mds_statfs to node 10.240.38.186@tcp failed: rc = -107
[ 936.180841] Lustre: lustre-MDT0000-mdc-ffff89618e2ab000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 936.183849] LustreError: 37840:0:(lmv_obd.c:1434:lmv_statfs()) lustre-MDT0000-mdc-ffff89618e2ab000: can't stat MDS #0: rc = -107
[ 936.208644] LustreError: 4441:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff8961cf3316c0 x1818604864572416/t4294967321(4294967321) o101->lustre-MDT0000-mdc-ffff89618e2ab000@10.240.38.186@tcp:12/10 lens 576/608 e 0 to 0 dl 1734357506 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'grep.0' uid:0 gid:0
[ 936.788315] Lustre: lustre-MDT0000-mdc-ffff89618e2ab000: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[ 936.809173] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[ 936.812759] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f1748a73 to 0x5d837553f1756215
[ 940.998752] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17n: run e2fsck against master\/slave MDT which contains remote dir ========================================================== 13:58:15 \(1734357495\)
[ 941.331640] Lustre: DEBUG MARKER: == sanity test 17n: run e2fsck against master/slave MDT which contains remote dir ========================================================== 13:58:15 (1734357495)
[ 941.703191] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_17n needs \>= 2 MDTs
[ 942.047313] Lustre: DEBUG MARKER: SKIP: sanity test_17n needs >= 2 MDTs
[ 942.453665] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17o: stat file with incompat LMA feature == 13:58:16 \(1734357496\)
[ 942.780209] Lustre: DEBUG MARKER: == sanity test 17o: stat file with incompat LMA feature == 13:58:16 (1734357496)
[ 948.071652] Lustre: 4443:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734357486/real 1734357486] req@ffff8961910e8000 x1818604865255168/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2ab000@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734357502 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 957.292650] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[ 957.295132] Lustre: lustre-MDT0000-mdc-ffff89618e2ab000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 957.298110] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f1756215 to 0x5d837553f17638c9
[ 957.300536] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[ 957.302151] Lustre: Skipped 1 previous similar message
[ 961.851203] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 962.967250] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[ 963.328013] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 963.431699] Lustre: 4442:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734357501/real 1734357501] req@ffff89618dd8d380 x1818604865636096/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2ab000@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734357517 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 963.630151] Lustre: DEBUG MARKER: lctl get_param -n at_max
[ 963.881652] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[ 964.310228] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 968.647496] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17p: symlink overwrite directory error message ========================================================== 13:58:42 \(1734357522\)
[ 969.246401] Lustre: DEBUG MARKER: == sanity test 17p: symlink overwrite directory error message ========================================================== 13:58:42 (1734357522)
[ 969.319711] Lustre: 4442:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734357506/real 1734357506] req@ffff89618dd8f400 x1818604865637248/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2ab000@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734357522 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 971.002685] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 18: touch ...\/f \; ls ... ======================================================================================== 13:58:45 \(1734357525\)
[ 971.336136] Lustre: DEBUG MARKER: == sanity test 18: touch .../f ; ls ... ======================================================================================== 13:58:45 (1734357525)
[ 972.956847] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19a: touch ...\/f19 \; ls -l ... \; rm ...\/f19 ===================================================================== 13:58:47 \(1734357527\)
[ 973.296284] Lustre: DEBUG MARKER: == sanity test 19a: touch .../f19 ; ls -l ... ; rm .../f19 ===================================================================== 13:58:47 (1734357527)
[ 975.025173] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19b: ls -l ...\/f19 \(should return error\) ======================================================================== 13:58:49 \(1734357529\)
[ 975.361968] Lustre: DEBUG MARKER: == sanity test 19b: ls -l .../f19 (should return error) ======================================================================== 13:58:49 (1734357529)
[ 976.971776] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19c: runas -u 500 -g 500 touch ...\/f19 \(should return error\) ============================================================ 13:58:51 \(1734357531\)
[ 977.303953] Lustre: DEBUG MARKER: == sanity test 19c: runas -u 500 -g 500 touch .../f19 (should return error) ============================================================ 13:58:51 (1734357531)
[ 978.924194] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19d: cat ...\/f19 \(should return error\) ======================================================================== 13:58:53 \(1734357533\)
[ 979.252756] Lustre: DEBUG MARKER: == sanity test 19d: cat .../f19 (should return error) ======================================================================== 13:58:53 (1734357533)
[ 980.870409] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 20: touch ...\/f \; ls -l ... =============== 13:58:55 \(1734357535\)
[ 981.202822] Lustre: DEBUG MARKER: == sanity test 20: touch .../f ; ls -l ... =============== 13:58:55 (1734357535)
[ 982.929851] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 21: write to dangling link ================ 13:58:57 \(1734357537\)
[ 983.258574] Lustre: DEBUG MARKER: == sanity test 21: write to dangling link ================ 13:58:57 (1734357537)
[ 984.923711] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 22: unpack tar archive as non-root user === 13:58:59 \(1734357539\)
[ 985.292265] Lustre: DEBUG MARKER: == sanity test 22: unpack tar archive as non-root user === 13:58:59 (1734357539)
[ 986.973266] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23a: O_CREAT\|O_EXCL in subdir ============= 13:59:01 \(1734357541\)
[ 987.340214] Lustre: DEBUG MARKER: == sanity test 23a: O_CREAT|O_EXCL in subdir ============= 13:59:01 (1734357541)
[ 988.991417] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23b: O_APPEND check ======================= 13:59:03 \(1734357543\)
[ 989.324354] Lustre: DEBUG MARKER: == sanity test 23b: O_APPEND check ======================= 13:59:03 (1734357543)
[ 990.998411] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23c: O_APPEND size checks for tiny writes ========================================================== 13:59:05 \(1734357545\)
[ 991.362630] Lustre: DEBUG MARKER: == sanity test 23c: O_APPEND size checks for tiny writes ========================================================== 13:59:05 (1734357545)
[ 993.973291] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23d: file offset is correct after appending writes ========================================================== 13:59:08 \(1734357548\)
[ 994.310002] Lustre: DEBUG MARKER: == sanity test 23d: file offset is correct after appending writes ========================================================== 13:59:08 (1734357548)
[ 995.946940] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24a: rename file to non-existent target === 13:59:10 \(1734357550\)
[ 996.276394] Lustre: DEBUG MARKER: == sanity test 24a: rename file to non-existent target === 13:59:10 (1734357550)
[ 997.934090] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24b: rename file to existing target ======= 13:59:12 \(1734357552\)
[ 998.264957] Lustre: DEBUG MARKER: == sanity test 24b: rename file to existing target ======= 13:59:12 (1734357552)
[ 999.931282] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24c: rename directory to non-existent target ========================================================== 13:59:14 \(1734357554\)
[ 1000.268146] Lustre: DEBUG MARKER: == sanity test 24c: rename directory to non-existent target ========================================================== 13:59:14 (1734357554)
[ 1001.941890] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24d: rename directory to existing target == 13:59:16 \(1734357556\)
[ 1002.282775] Lustre: DEBUG MARKER: == sanity test 24d: rename directory to existing target == 13:59:16 (1734357556)
[ 1003.946449] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24e: touch ...\/R5a\/f\; rename ...\/R5a\/f ...\/R5b\/g ================================================================ 13:59:18 \(1734357558\)
[ 1004.279960] Lustre: DEBUG MARKER: == sanity test 24e: touch .../R5a/f; rename .../R5a/f .../R5b/g ================================================================ 13:59:18 (1734357558)
[ 1005.949275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24f: touch ...\/R6a\/f R6b\/g\; mv ...\/R6a\/f ...\/R6b\/g ============================================================== 13:59:20 \(1734357560\)
[ 1006.280695] Lustre: DEBUG MARKER: == sanity test 24f: touch .../R6a/f R6b/g; mv .../R6a/f .../R6b/g ============================================================== 13:59:20 (1734357560)
[ 1007.977275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24g: mkdir ...\/R7{a,b}\/d\; mv ...\/R7a\/d ...\/R7b\/e ================================================================ 13:59:22 \(1734357562\)
[ 1008.311011] Lustre: DEBUG MARKER: == sanity test 24g: mkdir .../R7a/d; .../R7b/d; mv .../R7a/d .../R7b/e ================================================================ 13:59:22 (1734357562)
[ 1009.999746] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24h: mkdir ...\/R8{a,b}\/{d,e}\; rename ...\/R8a\/d ...\/R8b\/e ========================================================== 13:59:24 \(1734357564\)
[ 1010.342846] Lustre: DEBUG MARKER: == sanity test 24h: mkdir .../R8a/d; .../R8a/e; .../R8b/d; .../R8b/e; rename .../R8a/d .../R8b/e ========================================================== 13:59:24 (1734357564)
[ 1012.012509] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24i: rename file to dir error: touch f \; mkdir a \; rename f a ========================================================== 13:59:26 \(1734357566\)
[ 1012.345442] Lustre: DEBUG MARKER: == sanity test 24i: rename file to dir error: touch f ; mkdir a ; rename f a ========================================================== 13:59:26 (1734357566)
[ 1013.985013] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24j: source does not exist ====================================================================================== 13:59:28 \(1734357568\)
[ 1014.323153] Lustre: DEBUG MARKER: == sanity test 24j: source does not exist ====================================================================================== 13:59:28 (1734357568)
[ 1015.965992] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24k: touch ...\/R11a\/f\; mv ...\/R11a\/f ...\/R11a\/d ================================================================= 13:59:30 \(1734357570\)
[ 1016.302970] Lustre: DEBUG MARKER: == sanity test 24k: touch .../R11a/f; mv .../R11a/f .../R11a/d ================================================================= 13:59:30 (1734357570)
[ 1017.979352] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24l: Renaming a file to itself ================================================================================== 13:59:32 \(1734357572\)
[ 1018.318339] Lustre: DEBUG MARKER: == sanity test 24l: Renaming a file to itself ================================================================================== 13:59:32 (1734357572)
[ 1019.959591] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24m: Renaming a file to a hard link to itself =================================================================== 13:59:34 \(1734357574\)
[ 1020.291119] Lustre: DEBUG MARKER: == sanity test 24m: Renaming a file to a hard link to itself =================================================================== 13:59:34 (1734357574)
[ 1021.922076] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24n: Statting the old file after renaming \(Posix rename 2\) ========================================================== 13:59:36 \(1734357576\)
[ 1022.266905] Lustre: DEBUG MARKER: == sanity test 24n: Statting the old file after renaming (Posix rename 2) ========================================================== 13:59:36 (1734357576)
[ 1023.911995] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24o: rename of files during htree split === 13:59:38 \(1734357578\)
[ 1024.238505] Lustre: DEBUG MARKER: == sanity test 24o: rename of files during htree split === 13:59:38 (1734357578)
[ 1078.399793] Autotest: Test running for 15 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1101.764175] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24p: mkdir ...\/R12{a,b}\; rename ...\/R12a ...\/R12b ========================================================== 14:00:55 \(1734357655\)
[ 1102.099200] Lustre: DEBUG MARKER: == sanity test 24p: mkdir .../R12a; .../R12b; rename .../R12a .../R12b ========================================================== 14:00:55 (1734357655)
[ 1103.790699] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24q: mkdir ...\/R13{a,b}\; open R13b rename R13a R13b ============================================================= 14:00:57 \(1734357657\)
[ 1104.134934] Lustre: DEBUG MARKER: == sanity test 24q: mkdir .../R13a; .../R13b; open R13b rename R13a R13b ============================================================= 14:00:57 (1734357657)
[ 1105.821497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24r: mkdir ...\/R14a\/b\; rename ...\/R14a ...\/R14a\/b =============================================================== 14:00:59 \(1734357659\)
[ 1106.156962] Lustre: DEBUG MARKER: == sanity test 24r: mkdir .../R14a/b; rename .../R14a .../R14a/b =============================================================== 14:00:59 (1734357659)
[ 1107.805281] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24s: mkdir ...\/R15a\/b\/c\; rename ...\/R15a ...\/R15a\/b\/c =========================================================== 14:01:01 \(1734357661\)
[ 1108.138483] Lustre: DEBUG MARKER: == sanity test 24s: mkdir .../R15a/b/c; rename .../R15a .../R15a/b/c =========================================================== 14:01:01 (1734357661)
[ 1109.772710] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24t: mkdir ...\/R16a\/b\/c\; rename ...\/R16a\/b\/c ...\/R16a =========================================================== 14:01:03 \(1734357663\)
[ 1110.107336] Lustre: DEBUG MARKER: == sanity test 24t: mkdir .../R16a/b/c; rename .../R16a/b/c .../R16a =========================================================== 14:01:03 (1734357663)
[ 1111.768547] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24u: create stripe file =================== 14:01:05 \(1734357665\)
[ 1112.103220] Lustre: DEBUG MARKER: == sanity test 24u: create stripe file =================== 14:01:05 (1734357665)
[ 1114.183250] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24v: list large directory \(test hash collision, b=17560\) ========================================================== 14:01:08 \(1734357668\)
[ 1114.529427] Lustre: DEBUG MARKER: == sanity test 24v: list large directory (test hash collision, b=17560) ========================================================== 14:01:08 (1734357668)
[ 1115.271467] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1154.372879] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1158.781469] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1235.430155] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1239.091162] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24w: Reading a file larger than 4Gb ======= 14:03:13 \(1734357793\)
[ 1239.430751] Lustre: DEBUG MARKER: == sanity test 24w: Reading a file larger than 4Gb ======= 14:03:13 (1734357793)
[ 1241.082002] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24x: cross MDT rename\/link ================ 14:03:15 \(1734357795\)
[ 1241.409694] Lustre: DEBUG MARKER: == sanity test 24x: cross MDT rename/link ================ 14:03:15 (1734357795)
[ 1241.786263] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24x needs \>= 2 MDTs
[ 1242.119044] Lustre: DEBUG MARKER: SKIP: sanity test_24x needs >= 2 MDTs
[ 1242.510904] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24y: rename\/link on the same dir should succeed ========================================================== 14:03:16 \(1734357796\)
[ 1242.846710] Lustre: DEBUG MARKER: == sanity test 24y: rename/link on the same dir should succeed ========================================================== 14:03:16 (1734357796)
[ 1243.220306] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24y needs \>= 2 MDTs
[ 1243.548175] Lustre: DEBUG MARKER: SKIP: sanity test_24y needs >= 2 MDTs
[ 1243.948146] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24z: cross-MDT rename is done as cp ======= 14:03:18 \(1734357798\)
[ 1244.277475] Lustre: DEBUG MARKER: == sanity test 24z: cross-MDT rename is done as cp ======= 14:03:18 (1734357798)
[ 1244.645667] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24z needs \>= 2 MDTs
[ 1244.975900] Lustre: DEBUG MARKER: SKIP: sanity test_24z needs >= 2 MDTs
[ 1245.375055] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24A: readdir\(\) returns correct number of entries. ========================================================== 14:03:19 \(1734357799\)
[ 1245.707988] Lustre: DEBUG MARKER: == sanity test 24A: readdir() returns correct number of entries. ========================================================== 14:03:19 (1734357799)
[ 1246.413866] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1249.009382] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1254.442150] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1257.077311] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1260.429884] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24B: readdir for striped dir return correct number of entries ========================================================== 14:03:34 \(1734357814\)
[ 1260.770615] Lustre: DEBUG MARKER: == sanity test 24B: readdir for striped dir return correct number of entries ========================================================== 14:03:34 (1734357814)
[ 1261.156138] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24B needs \>= 2 MDTs
[ 1261.486141] Lustre: DEBUG MARKER: SKIP: sanity test_24B needs >= 2 MDTs
[ 1261.889518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24C: check .. in striped dir ============== 14:03:36 \(1734357816\)
[ 1262.238983] Lustre: DEBUG MARKER: == sanity test 24C: check .. in striped dir ============== 14:03:36 (1734357816)
[ 1262.601145] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24C needs \>= 2 MDTs
[ 1262.929737] Lustre: DEBUG MARKER: SKIP: sanity test_24C needs >= 2 MDTs
[ 1263.335755] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24E: cross MDT rename\/link ================ 14:03:37 \(1734357817\)
[ 1263.663938] Lustre: DEBUG MARKER: == sanity test 24E: cross MDT rename/link ================ 14:03:37 (1734357817)
[ 1264.033461] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24E needs \>= 4 MDTs
[ 1264.371274] Lustre: DEBUG MARKER: SKIP: sanity test_24E needs >= 4 MDTs
[ 1264.827992] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24F: hash order vs readdir \(LU-11330\) ===== 14:03:38 \(1734357818\)
[ 1265.163050] Lustre: DEBUG MARKER: == sanity test 24F: hash order vs readdir (LU-11330) ===== 14:03:38 (1734357818)
[ 1265.537936] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24F needs \>= 2 MDTs
[ 1265.862123] Lustre: DEBUG MARKER: SKIP: sanity test_24F needs >= 2 MDTs
[ 1266.265829] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24G: migrate symlink in rename ============ 14:03:40 \(1734357820\)
[ 1266.600255] Lustre: DEBUG MARKER: == sanity test 24G: migrate symlink in rename ============ 14:03:40 (1734357820)
[ 1266.975139] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24G needs \>= 2 MDTs
[ 1267.308053] Lustre: DEBUG MARKER: SKIP: sanity test_24G needs >= 2 MDTs
[ 1267.736510] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24H: repeat FLD_QUERY rpc ================= 14:03:41 \(1734357821\)
[ 1268.074420] Lustre: DEBUG MARKER: == sanity test 24H: repeat FLD_QUERY rpc ================= 14:03:41 (1734357821)
[ 1268.439146] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_24H needs \>= 2 MDTs
[ 1268.771785] Lustre: DEBUG MARKER: SKIP: sanity test_24H needs >= 2 MDTs
[ 1269.184203] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 25a: create file in symlinked directory ========================================================================= 14:03:43 \(1734357823\)
[ 1269.519954] Lustre: DEBUG MARKER: == sanity test 25a: create file in symlinked directory ========================================================================= 14:03:43 (1734357823)
[ 1271.188367] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 25b: lookup file in symlinked directory ========================================================================= 14:03:45 \(1734357825\)
[ 1271.523229] Lustre: DEBUG MARKER: == sanity test 25b: lookup file in symlinked directory ========================================================================= 14:03:45 (1734357825)
[ 1273.132043] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26a: multiple component symlink ================================================================================= 14:03:47 \(1734357827\)
[ 1273.464085] Lustre: DEBUG MARKER: == sanity test 26a: multiple component symlink ================================================================================= 14:03:47 (1734357827)
[ 1275.110759] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26b: multiple component symlink at end of lookup ================================================================ 14:03:49 \(1734357829\)
[ 1275.445017] Lustre: DEBUG MARKER: == sanity test 26b: multiple component symlink at end of lookup ================================================================ 14:03:49 (1734357829)
[ 1277.091183] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26c: chain of symlinks ==================== 14:03:51 \(1734357831\)
[ 1277.418569] Lustre: DEBUG MARKER: == sanity test 26c: chain of symlinks ==================== 14:03:51 (1734357831)
[ 1279.090958] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26d: create multiple component recursive symlink ========================================================== 14:03:53 \(1734357833\)
[ 1279.422950] Lustre: DEBUG MARKER: == sanity test 26d: create multiple component recursive symlink ========================================================== 14:03:53 (1734357833)
[ 1281.029061] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26e: unlink multiple component recursive symlink ========================================================== 14:03:55 \(1734357835\)
[ 1281.352337] Lustre: DEBUG MARKER: == sanity test 26e: unlink multiple component recursive symlink ========================================================== 14:03:55 (1734357835)
[ 1282.961705] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26f: rm -r of a directory which has recursive symlink ========================================================== 14:03:57 \(1734357837\)
[ 1283.305857] Lustre: DEBUG MARKER: == sanity test 26f: rm -r of a directory which has recursive symlink ========================================================== 14:03:57 (1734357837)
[ 1285.097306] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27a: one stripe file ====================== 14:03:59 \(1734357839\)
[ 1285.430809] Lustre: DEBUG MARKER: == sanity test 27a: one stripe file ====================== 14:03:59 (1734357839)
[ 1287.083245] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27b: create and write to two stripe file == 14:04:01 \(1734357841\)
[ 1287.411248] Lustre: DEBUG MARKER: == sanity test 27b: create and write to two stripe file == 14:04:01 (1734357841)
[ 1289.071838] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27ca: one stripe on specified OST ========= 14:04:03 \(1734357843\)
[ 1289.399467] Lustre: DEBUG MARKER: == sanity test 27ca: one stripe on specified OST ========= 14:04:03 (1734357843)
[ 1291.136772] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cb: two stripes on specified OSTs ======= 14:04:05 \(1734357845\)
[ 1291.471452] Lustre: DEBUG MARKER: == sanity test 27cb: two stripes on specified OSTs ======= 14:04:05 (1734357845)
[ 1293.225593] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cc: two stripes on the same OST ========= 14:04:07 \(1734357847\)
[ 1293.560429] Lustre: DEBUG MARKER: == sanity test 27cc: two stripes on the same OST ========= 14:04:07 (1734357847)
[ 1295.330025] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cd: four stripes on two OSTs ============ 14:04:09 \(1734357849\)
[ 1295.664156] Lustre: DEBUG MARKER: == sanity test 27cd: four stripes on two OSTs ============ 14:04:09 (1734357849)
[ 1297.430115] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27ce: more stripes than OSTs with -o ====== 14:04:11 \(1734357851\)
[ 1297.759499] Lustre: DEBUG MARKER: == sanity test 27ce: more stripes than OSTs with -o ====== 14:04:11 (1734357851)
[ 1299.526843] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cf: \'setstripe -o\' on inactive OSTs should return error ========================================================== 14:04:13 \(1734357853\)
[ 1299.857132] Lustre: DEBUG MARKER: == sanity test 27cf: 'setstripe -o' on inactive OSTs should return error ========================================================== 14:04:13 (1734357853)
[ 1304.278616] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cg: 1000 shouldn\'t cause too many credits ========================================================== 14:04:18 \(1734357858\)
[ 1304.616610] Lustre: DEBUG MARKER: == sanity test 27cg: 1000 shouldn't cause too many credits ========================================================== 14:04:18 (1734357858)
[ 1307.971237] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27d: create file with default settings ==== 14:04:22 \(1734357862\)
[ 1308.319427] Lustre: DEBUG MARKER: == sanity test 27d: create file with default settings ==== 14:04:22 (1734357862)
[ 1309.992792] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27e: setstripe existing file \(should return error\) ========================================================== 14:04:24 \(1734357864\)
[ 1310.328337] Lustre: DEBUG MARKER: == sanity test 27e: setstripe existing file (should return error) ========================================================== 14:04:24 (1734357864)
[ 1312.093116] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27f: setstripe with bad stripe size \(should return error\) ========================================================== 14:04:26 \(1734357866\)
[ 1312.425805] Lustre: DEBUG MARKER: == sanity test 27f: setstripe with bad stripe size (should return error) ========================================================== 14:04:26 (1734357866)
[ 1314.080847] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27g: \/usr\/bin\/lfs getstripe with no objects ========================================================== 14:04:28 \(1734357868\)
[ 1314.414440] Lustre: DEBUG MARKER: == sanity test 27g: /usr/bin/lfs getstripe with no objects ========================================================== 14:04:28 (1734357868)
[ 1316.095584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27ga: \/usr\/bin\/lfs getstripe with missing file \(should return error\) ========================================================== 14:04:30 \(1734357870\)
[ 1316.430991] Lustre: DEBUG MARKER: == sanity test 27ga: /usr/bin/lfs getstripe with missing file (should return error) ========================================================== 14:04:30 (1734357870)
[ 1318.104597] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27i: \/usr\/bin\/lfs getstripe with some objects ========================================================== 14:04:32 \(1734357872\)
[ 1318.437173] Lustre: DEBUG MARKER: == sanity test 27i: /usr/bin/lfs getstripe with some objects ========================================================== 14:04:32 (1734357872)
[ 1320.093186] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27j: setstripe with bad stripe offset \(should return error\) ========================================================== 14:04:34 \(1734357874\)
[ 1320.455223] Lustre: DEBUG MARKER: == sanity test 27j: setstripe with bad stripe offset (should return error) ========================================================== 14:04:34 (1734357874)
[ 1322.118077] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27k: limit i_blksize for broken user apps ========================================================== 14:04:36 \(1734357876\)
[ 1322.448677] Lustre: DEBUG MARKER: == sanity test 27k: limit i_blksize for broken user apps ========================================================== 14:04:36 (1734357876)
[ 1324.140083] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27l: check setstripe permissions \(should return error\) ========================================================== 14:04:38 \(1734357878\)
[ 1324.472534] Lustre: DEBUG MARKER: == sanity test 27l: check setstripe permissions (should return error) ========================================================== 14:04:38 (1734357878)
[ 1326.078378] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_27m skipping SLOW test 27m
[ 1326.409211] Lustre: DEBUG MARKER: SKIP: sanity test_27m skipping SLOW test 27m
[ 1326.801682] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27n: create file with some full OSTs ====== 14:04:40 \(1734357880\)
[ 1327.135303] Lustre: DEBUG MARKER: == sanity test 27n: create file with some full OSTs ====== 14:04:40 (1734357880)
[ 1348.533550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27o: create file with all full OSTs \(should error\) ========================================================== 14:05:02 \(1734357902\)
[ 1348.863265] Lustre: DEBUG MARKER: == sanity test 27o: create file with all full OSTs (should error) ========================================================== 14:05:02 (1734357902)
[ 1379.176062] Autotest: Test running for 20 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1384.884444] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27oo: don\'t let few threads to reserve too many objects ========================================================== 14:05:39 \(1734357939\)
[ 1385.229655] Lustre: DEBUG MARKER: == sanity test 27oo: don't let few threads to reserve too many objects ========================================================== 14:05:39 (1734357939)
[ 1407.850731] Lustre: lustre-OST0000-osc-ffff89618e2ab000: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1416.847139] Lustre: lustre-OST0000-osc-ffff89618e2ab000: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[ 1416.853557] Lustre: Skipped 1 previous similar message
[ 1420.335784] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27p: append to a truncated file with some full OSTs ========================================================== 14:06:14 \(1734357974\)
[ 1420.674814] Lustre: DEBUG MARKER: == sanity test 27p: append to a truncated file with some full OSTs ========================================================== 14:06:14 (1734357974)
[ 1438.569227] Lustre: lustre-OST0000-osc-ffff89618e2ab000: disconnect after 22s idle
[ 1451.136766] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27q: append to truncated file with all OSTs full \(should error\) ========================================================== 14:06:45 \(1734358005\)
[ 1451.468487] Lustre: DEBUG MARKER: == sanity test 27q: append to truncated file with all OSTs full (should error) ========================================================== 14:06:45 (1734358005)
[ 1487.274509] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27r: stripe file with some full OSTs \(shouldn\'t LBUG\) =========================================================== 14:07:21 \(1734358041\)
[ 1487.600044] Lustre: DEBUG MARKER: == sanity test 27r: stripe file with some full OSTs (shouldn't LBUG) =========================================================== 14:07:21 (1734358041)
[ 1508.795044] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27s: lsm_xfersize overflow \(should error\) \(bug 10725\) ========================================================== 14:07:42 \(1734358062\)
[ 1509.130827] Lustre: DEBUG MARKER: == sanity test 27s: lsm_xfersize overflow (should error) (bug 10725) ========================================================== 14:07:42 (1734358062)
[ 1510.249355] Lustre: lustre-OST0002-osc-ffff89618e2ab000: disconnect after 21s idle
[ 1510.806514] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27t: check that utils parse path correctly ========================================================== 14:07:44 \(1734358064\)
[ 1511.148117] Lustre: DEBUG MARKER: == sanity test 27t: check that utils parse path correctly ========================================================== 14:07:44 (1734358064)
[ 1512.849106] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27u: skip object creation on OSC w\/o objects ========================================================== 14:07:46 \(1734358066\)
[ 1513.189558] Lustre: DEBUG MARKER: == sanity test 27u: skip object creation on OSC w/o objects ========================================================== 14:07:46 (1734358066)
[ 1514.333731] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1515.369269] Lustre: lustre-OST0004-osc-ffff89618e2ab000: disconnect after 24s idle
[ 1515.370718] Lustre: Skipped 1 previous similar message
[ 1515.920498] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1520.042241] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1521.090297] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1530.729449] Lustre: lustre-OST0005-osc-ffff89618e2ab000: disconnect after 20s idle
[ 1533.858393] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27v: skip object creation on slow OST ===== 14:08:07 \(1734358087\)
[ 1534.271104] Lustre: DEBUG MARKER: == sanity test 27v: skip object creation on slow OST ===== 14:08:07 (1734358087)
[ 1584.070054] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27w: check \/usr\/bin\/lfs setstripe -S and getstrip -d options ========================================================== 14:08:58 \(1734358138\)
[ 1584.413699] Lustre: DEBUG MARKER: == sanity test 27w: check /usr/bin/lfs setstripe -S and getstrip -d options ========================================================== 14:08:58 (1734358138)
[ 1586.080915] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27wa: check \/usr\/bin\/lfs setstripe -c -i options ========================================================== 14:09:00 \(1734358140\)
[ 1586.421133] Lustre: DEBUG MARKER: == sanity test 27wa: check /usr/bin/lfs setstripe -c -i options ========================================================== 14:09:00 (1734358140)
[ 1587.049549] Lustre: lustre-OST0000-osc-ffff89618e2ab000: disconnect after 20s idle
[ 1587.051014] Lustre: Skipped 2 previous similar messages
[ 1588.211077] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27x: create files while OST0 is degraded == 14:09:02 \(1734358142\)
[ 1588.542758] Lustre: DEBUG MARKER: == sanity test 27x: create files while OST0 is degraded == 14:09:02 (1734358142)
[ 1601.631095] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27y: create files while OST0 is degraded and the rest inactive ========================================================== 14:09:15 \(1734358155\)
[ 1601.967053] Lustre: DEBUG MARKER: == sanity test 27y: create files while OST0 is degraded and the rest inactive ========================================================== 14:09:15 (1734358155)
[ 1630.994677] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27z: check SEQ\/OID on the MDT and OST filesystems ========================================================== 14:09:45 \(1734358185\)
[ 1631.386514] Lustre: DEBUG MARKER: == sanity test 27z: check SEQ/OID on the MDT and OST filesystems ========================================================== 14:09:45 (1734358185)
[ 1642.396360] Lustre: DEBUG MARKER: /usr/sbin/lctl mark check file \/mnt\/lustre\/d27z.sanity\/f27z.sanity-1
[ 1642.813037] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-1
[ 1643.221775] Lustre: DEBUG MARKER: /usr/sbin/lctl mark FID seq 0x200000401, oid 0x1cdd5 ver 0x0
[ 1643.557074] Lustre: DEBUG MARKER: FID seq 0x200000401, oid 0x1cdd5 ver 0x0
[ 1643.932446] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LOV seq 0x200000401, oid 0x1cdd5, count: 1
[ 1644.270899] Lustre: DEBUG MARKER: LOV seq 0x200000401, oid 0x1cdd5, count: 1
[ 1644.651299] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:0 ost:0 oid:426\/0x1aa seq:0x240000400
[ 1644.976197] Lustre: DEBUG MARKER: want: stripe:0 ost:0 oid:426/0x1aa seq:0x240000400
[ 1646.088705] Lustre: DEBUG MARKER: /usr/sbin/lctl mark check file \/mnt\/lustre\/d27z.sanity\/f27z.sanity-2
[ 1646.427900] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-2
[ 1646.833317] Lustre: DEBUG MARKER: /usr/sbin/lctl mark FID seq 0x200000401, oid 0x1cdd7 ver 0x0
[ 1647.165187] Lustre: DEBUG MARKER: FID seq 0x200000401, oid 0x1cdd7 ver 0x0
[ 1647.549201] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LOV seq 0x200000401, oid 0x1cdd7, count: 7
[ 1647.878759] Lustre: DEBUG MARKER: LOV seq 0x200000401, oid 0x1cdd7, count: 7
[ 1648.274714] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:0 ost:6 oid:399\/0x18f seq:0x3c0000400
[ 1648.606113] Lustre: DEBUG MARKER: want: stripe:0 ost:6 oid:399/0x18f seq:0x3c0000400
[ 1649.537534] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:1 ost:0 oid:427\/0x1ab seq:0x240000400
[ 1649.875213] Lustre: DEBUG MARKER: want: stripe:1 ost:0 oid:427/0x1ab seq:0x240000400
[ 1650.784373] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:2 ost:1 oid:504\/0x1f8 seq:0x280000400
[ 1651.119205] Lustre: DEBUG MARKER: want: stripe:2 ost:1 oid:504/0x1f8 seq:0x280000400
[ 1652.108054] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:3 ost:2 oid:385\/0x181 seq:0x2c0000400
[ 1652.445974] Lustre: DEBUG MARKER: want: stripe:3 ost:2 oid:385/0x181 seq:0x2c0000400
[ 1653.568928] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:4 ost:3 oid:403\/0x193 seq:0x300000400
[ 1653.905955] Lustre: DEBUG MARKER: want: stripe:4 ost:3 oid:403/0x193 seq:0x300000400
[ 1654.723010] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:5 ost:4 oid:403\/0x193 seq:0x340000400
[ 1655.054993] Lustre: DEBUG MARKER: want: stripe:5 ost:4 oid:403/0x193 seq:0x340000400
[ 1655.916698] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:6 ost:5 oid:404\/0x194 seq:0x380000400
[ 1656.239050] Lustre: DEBUG MARKER: want: stripe:6 ost:5 oid:404/0x194 seq:0x380000400
[ 1658.383655] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27A: check filesystem-wide default LOV EA values ========================================================== 14:10:12 \(1734358212\)
[ 1658.716019] Lustre: DEBUG MARKER: == sanity test 27A: check filesystem-wide default LOV EA values ========================================================== 14:10:12 (1734358212)
[ 1658.870150] Lustre: DEBUG MARKER: /usr/bin/lfs getstripe -c /mnt/lustre | sed 's/ *//g'
[ 1660.761370] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27B: call setstripe on open unlinked file\/rename victim ========================================================== 14:10:14 \(1734358214\)
[ 1661.101711] Lustre: DEBUG MARKER: == sanity test 27B: call setstripe on open unlinked file/rename victim ========================================================== 14:10:14 (1734358214)
[ 1662.795151] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ca: check full striping across all OSTs ========================================================== 14:10:16 \(1734358216\)
[ 1663.127055] Lustre: DEBUG MARKER: == sanity test 27Ca: check full striping across all OSTs ========================================================== 14:10:16 (1734358216)
[ 1665.044438] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cb: more stripes than OSTs with -C ====== 14:10:19 \(1734358219\)
[ 1665.378417] Lustre: DEBUG MARKER: == sanity test 27Cb: more stripes than OSTs with -C ====== 14:10:19 (1734358219)
[ 1667.158955] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cc: fewer stripes than OSTs does not set overstriping ========================================================== 14:10:21 \(1734358221\)
[ 1667.489726] Lustre: DEBUG MARKER: == sanity test 27Cc: fewer stripes than OSTs does not set overstriping ========================================================== 14:10:21 (1734358221)
[ 1669.260714] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cd: test maximum stripe count =========== 14:10:23 \(1734358223\)
[ 1669.623198] Lustre: DEBUG MARKER: == sanity test 27Cd: test maximum stripe count =========== 14:10:23 (1734358223)
[ 1680.000876] Autotest: Test running for 25 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1692.874078] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ce: test pool with overstriping ========= 14:10:47 \(1734358247\)
[ 1693.214055] Lustre: DEBUG MARKER: == sanity test 27Ce: test pool with overstriping ========= 14:10:47 (1734358247)
[ 1695.732266] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ce 2>/dev/null || echo foo
[ 1698.139709] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ce |
[ 1698.139709] grep -e lustre-OST0000_UUID | sort -u | tr '\n' ' '
[ 1705.387542] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ce 2>/dev/null || echo foo
[ 1705.679710] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cf: test default inheritance with overstriping ========================================================== 14:10:59 \(1734358259\)
[ 1706.019595] Lustre: DEBUG MARKER: == sanity test 27Cf: test default inheritance with overstriping ========================================================== 14:10:59 (1734358259)
[ 1708.392706] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cg: test setstripe with wrong OST idx === 14:11:02 \(1734358262\)
[ 1708.720625] Lustre: DEBUG MARKER: == sanity test 27Cg: test setstripe with wrong OST idx === 14:11:02 (1734358262)
[ 1710.368960] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ci: add an overstriping component ======= 14:11:04 \(1734358264\)
[ 1710.709006] Lustre: DEBUG MARKER: == sanity test 27Ci: add an overstriping component ======= 14:11:04 (1734358264)
[ 1716.884437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cj: overstriping with -C for max values in multiple of targets ========================================================== 14:11:11 \(1734358271\)
[ 1717.226054] Lustre: DEBUG MARKER: == sanity test 27Cj: overstriping with -C for max values in multiple of targets ========================================================== 14:11:11 (1734358271)
[ 1719.015421] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27D: validate llapi_layout API ============ 14:11:13 \(1734358273\)
[ 1719.344718] Lustre: DEBUG MARKER: == sanity test 27D: validate llapi_layout API ============ 14:11:13 (1734358273)
[ 1721.879613] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 2>/dev/null || echo foo
[ 1724.273599] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool |
[ 1724.273599] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID | sort -u | tr '\n' ' '
[ 1739.664818] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 2>/dev/null || echo foo
[ 1741.291279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27E: check that default extended attribute size properly increases ========================================================== 14:11:35 \(1734358295\)
[ 1741.620031] Lustre: DEBUG MARKER: == sanity test 27E: check that default extended attribute size properly increases ========================================================== 14:11:35 (1734358295)
[ 1743.316410] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27F: Client resend delayed layout creation with non-zero size ========================================================== 14:11:37 \(1734358297\)
[ 1743.658603] Lustre: DEBUG MARKER: == sanity test 27F: Client resend delayed layout creation with non-zero size ========================================================== 14:11:37 (1734358297)
[ 1745.772087] Lustre: lustre-OST0000-osc-ffff89618e2ab000: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1750.891819] Lustre: lustre-OST0002-osc-ffff89618e2ab000: Connection to lustre-OST0002 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1750.894687] Lustre: Skipped 1 previous similar message
[ 1763.925389] LustreError: lustre-MDT0000-mdc-ffff89618e2ab000: operation ldlm_enqueue to node 10.240.38.186@tcp failed: rc = -107
[ 1763.927436] Lustre: lustre-MDT0000-mdc-ffff89618e2ab000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1763.930218] Lustre: Skipped 2 previous similar messages
[ 1763.934138] Lustre: lustre-MDT0000-mdc-ffff89618e2ab000: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[ 1777.514041] Lustre: 4443:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734358315/real 1734358315] req@ffff8961d2ce5040 x1818604946596224/t0(0) o400->lustre-OST0006-osc-ffff89618e2ab000@10.240.45.50@tcp:28/4 lens 224/224 e 0 to 1 dl 1734358331 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 1787.960871] Lustre: lustre-OST0002-osc-ffff89618e2ab000: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[ 1797.537315] Lustre: lustre-OST0004-osc-ffff89618e2ab000: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[ 1797.539212] Lustre: Skipped 3 previous similar messages
[ 1805.162190] Lustre: lustre-OST0001-osc-ffff89618e2ab000: disconnect after 23s idle
[ 1814.568741] Lustre: lustre-OST0006-osc-ffff89618e2ab000: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[ 1814.570677] Lustre: Skipped 1 previous similar message
[ 1819.451018] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27G: Clear OST pool from stripe =========== 14:12:53 \(1734358373\)
[ 1819.790047] Lustre: DEBUG MARKER: == sanity test 27G: Clear OST pool from stripe =========== 14:12:53 (1734358373)
[ 1822.995429] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 2>/dev/null || echo foo
[ 1825.433608] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool |
[ 1825.433608] grep -e lustre-OST0000_UUID | sort -u | tr '\n' ' '
[ 1833.194756] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 2>/dev/null || echo foo
[ 1833.484244] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27H: Set specific OSTs stripe ============= 14:13:07 \(1734358387\)
[ 1833.822675] Lustre: DEBUG MARKER: == sanity test 27H: Set specific OSTs stripe ============= 14:13:07 (1734358387)
[ 1836.230701] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27I: check that root dir striping does not break parent dir one ========================================================== 14:13:10 \(1734358390\)
[ 1836.562810] Lustre: DEBUG MARKER: == sanity test 27I: check that root dir striping does not break parent dir one ========================================================== 14:13:10 (1734358390)
[ 1839.077973] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27I 2>/dev/null || echo foo
[ 1841.474416] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27I |
[ 1841.474416] grep -e lustre-OST0001_UUID | sort -u | tr '\n' ' '
[ 1848.627861] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27I 2>/dev/null || echo foo
[ 1848.921241] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ia: check that root dir pool is dropped with conflict parent dir settings ========================================================== 14:13:23 \(1734358403\)
[ 1849.260011] Lustre: DEBUG MARKER: == sanity test 27Ia: check that root dir pool is dropped with conflict parent dir settings ========================================================== 14:13:23 (1734358403)
[ 1851.770427] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ia 2>/dev/null || echo foo
[ 1854.167176] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ia |
[ 1854.167176] grep -e lustre-OST0001_UUID | sort -u | tr '\n' ' '
[ 1861.455901] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ia 2>/dev/null || echo foo
[ 1861.751542] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27J: basic ops on file with foreign LOV === 14:13:35 \(1734358415\)
[ 1862.086051] Lustre: DEBUG MARKER: == sanity test 27J: basic ops on file with foreign LOV === 14:13:35 (1734358415)
[ 1863.922867] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27K: basic ops on dir with foreign LMV ==== 14:13:38 \(1734358418\)
[ 1864.258962] Lustre: DEBUG MARKER: == sanity test 27K: basic ops on dir with foreign LMV ==== 14:13:38 (1734358418)
[ 1866.060674] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27L: lfs pool_list gives correct pool name ========================================================== 14:13:40 \(1734358420\)
[ 1866.399061] Lustre: DEBUG MARKER: == sanity test 27L: lfs pool_list gives correct pool name ========================================================== 14:13:40 (1734358420)
[ 1868.926212] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27L 2>/dev/null || echo foo
[ 1873.638751] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27L 2>/dev/null || echo foo
[ 1873.935294] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27M: test O_APPEND striping =============== 14:13:48 \(1734358428\)
[ 1874.276804] Lustre: DEBUG MARKER: == sanity test 27M: test O_APPEND striping =============== 14:13:48 (1734358428)
[ 1881.635628] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27M 2>/dev/null || echo foo
[ 1884.035046] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27M |
[ 1884.035046] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID | sort -u | tr '\n' ' '
[ 1893.790200] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27M 2>/dev/null || echo foo
[ 1895.492254] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27N: lctl pool_list on separate MGS gives correct pool name ========================================================== 14:14:09 \(1734358449\)
[ 1895.828323] Lustre: DEBUG MARKER: == sanity test 27N: lctl pool_list on separate MGS gives correct pool name ========================================================== 14:14:09 (1734358449)
[ 1896.222793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_27N needs separate MGS\/MDT
[ 1896.561953] Lustre: DEBUG MARKER: SKIP: sanity test_27N needs separate MGS/MDT
[ 1896.970493] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27O: basic ops on foreign file of symlink type ========================================================== 14:14:11 \(1734358451\)
[ 1897.311357] Lustre: DEBUG MARKER: == sanity test 27O: basic ops on foreign file of symlink type ========================================================== 14:14:11 (1734358451)
[ 1899.544855] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27P: basic ops on foreign dir of foreign_symlink type ========================================================== 14:14:13 \(1734358453\)
[ 1899.884379] Lustre: DEBUG MARKER: == sanity test 27P: basic ops on foreign dir of foreign_symlink type ========================================================== 14:14:13 (1734358453)
[ 1902.061472] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Q: llapi_file_get_stripe\(\) works on symlinks ========================================================== 14:14:16 \(1734358456\)
[ 1902.395030] Lustre: DEBUG MARKER: == sanity test 27Q: llapi_file_get_stripe() works on symlinks ========================================================== 14:14:16 (1734358456)
[ 1904.104704] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27R: test max_stripecount limitation when stripe count is set to -1 ========================================================== 14:14:18 \(1734358458\)
[ 1904.428876] Lustre: DEBUG MARKER: == sanity test 27R: test max_stripecount limitation when stripe count is set to -1 ========================================================== 14:14:18 (1734358458)
[ 1907.455205] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27T: no eio on close on partial write due to enosp ========================================================== 14:14:21 \(1734358461\)
[ 1907.795371] Lustre: DEBUG MARKER: == sanity test 27T: no eio on close on partial write due to enosp ========================================================== 14:14:21 (1734358461)
[ 1910.086785] Lustre: *** cfs_fail_loc=411, val=1***
[ 1913.351527] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27U: append pool and stripe count work with composite default layout ========================================================== 14:14:27 \(1734358467\)
[ 1913.701789] Lustre: DEBUG MARKER: == sanity test 27U: append pool and stripe count work with composite default layout ========================================================== 14:14:27 (1734358467)
[ 1916.945363] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-append 2>/dev/null || echo foo
[ 1919.340026] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-append |
[ 1919.340026] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID | sort -u | tr '\n' ' '
[ 1921.717732] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-normal 2>/dev/null || echo foo
[ 1924.126301] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-normal |
[ 1924.126301] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID | sort -u | tr '\n' ' '
[ 1928.042675] Lustre: lustre-OST0003-osc-ffff89618e2ab000: disconnect after 24s idle
[ 1928.044120] Lustre: Skipped 9 previous similar messages
[ 1934.603184] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-append 2>/dev/null || echo foo
[ 1942.828272] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-normal 2>/dev/null || echo foo
[ 1943.921669] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27V: creating widely striped file races with deactivating OST ========================================================== 14:14:58 \(1734358498\)
[ 1944.341070] Lustre: DEBUG MARKER: == sanity test 27V: creating widely striped file races with deactivating OST ========================================================== 14:14:58 (1734358498)
[ 1950.062090] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27W: test enable_setstripe_gid ============ 14:15:04 \(1734358504\)
[ 1950.404380] Lustre: DEBUG MARKER: == sanity test 27W: test enable_setstripe_gid ============ 14:15:04 (1734358504)
[ 1952.237245] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 28: create\/mknod\/mkdir with bad file types ====================================================================== 14:15:06 \(1734358506\)
[ 1952.570522] Lustre: DEBUG MARKER: == sanity test 28: create/mknod/mkdir with bad file types ====================================================================== 14:15:06 (1734358506)
[ 1954.409493] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 29: IT_GETATTR regression ====================================================================================== 14:15:08 \(1734358508\)
[ 1954.737078] Lustre: DEBUG MARKER: == sanity test 29: IT_GETATTR regression ====================================================================================== 14:15:08 (1734358508)
[ 1956.494765] Lustre: DEBUG MARKER: /usr/sbin/lctl mark first d29
[ 1956.845516] Lustre: DEBUG MARKER: first d29
[ 1957.249173] Lustre: DEBUG MARKER: /usr/sbin/lctl mark second d29
[ 1957.596449] Lustre: DEBUG MARKER: second d29
[ 1957.993436] Lustre: DEBUG MARKER: /usr/sbin/lctl mark done
[ 1958.328370] Lustre: DEBUG MARKER: done
[ 1960.001104] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30a: execute binary from Lustre \(execve\) ======================================================================== 14:15:14 \(1734358514\)
[ 1960.334911] Lustre: DEBUG MARKER: == sanity test 30a: execute binary from Lustre (execve) ======================================================================== 14:15:14 (1734358514)
[ 1962.018055] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30b: execute binary from Lustre as non-root ===================================================================== 14:15:16 \(1734358516\)
[ 1962.356109] Lustre: DEBUG MARKER: == sanity test 30b: execute binary from Lustre as non-root ===================================================================== 14:15:16 (1734358516)
[ 1964.062639] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30c: execute binary from Lustre without read perms ============================================================== 14:15:18 \(1734358518\)
[ 1964.395976] Lustre: DEBUG MARKER: == sanity test 30c: execute binary from Lustre without read perms ============================================================== 14:15:18 (1734358518)
[ 1966.409049] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30d: execute binary from Lustre while clear locks ========================================================== 14:15:20 \(1734358520\)
[ 1966.736676] Lustre: DEBUG MARKER: == sanity test 30d: execute binary from Lustre while clear locks ========================================================== 14:15:20 (1734358520)
[ 1980.814962] Autotest: Test running for 30 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2071.402953] Lustre: lustre-OST0006-osc-ffff89618e2ab000: disconnect after 24s idle
[ 2071.409067] Lustre: Skipped 10 previous similar messages
[ 2104.872675] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31a: open-unlink file ============================================================================================ 14:17:39 \(1734358659\)
[ 2105.207796] Lustre: DEBUG MARKER: == sanity test 31a: open-unlink file ============================================================================================ 14:17:39 (1734358659)
[ 2115.004400] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31b: unlink file with multiple links while open ================================================================= 14:17:49 \(1734358669\)
[ 2115.334294] Lustre: DEBUG MARKER: == sanity test 31b: unlink file with multiple links while open ================================================================= 14:17:49 (1734358669)
[ 2117.476594] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31c: open-unlink file with multiple links ======================================================================= 14:17:51 \(1734358671\)
[ 2117.813247] Lustre: DEBUG MARKER: == sanity test 31c: open-unlink file with multiple links ======================================================================= 14:17:51 (1734358671)
[ 2119.494634] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31d: remove of open directory =================================================================================== 14:17:53 \(1734358673\)
[ 2119.826181] Lustre: DEBUG MARKER: == sanity test 31d: remove of open directory =================================================================================== 14:17:53 (1734358673)
[ 2121.502548] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31e: remove of open non-empty directory ========================================================================= 14:17:55 \(1734358675\)
[ 2121.836552] Lustre: DEBUG MARKER: == sanity test 31e: remove of open non-empty directory ========================================================================= 14:17:55 (1734358675)
[ 2123.494386] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31f: remove of open directory with open-unlink file ============================================================= 14:17:57 \(1734358677\)
[ 2123.831333] Lustre: DEBUG MARKER: == sanity test 31f: remove of open directory with open-unlink file ============================================================= 14:17:57 (1734358677)
[ 2131.617438] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31g: cross directory link================== 14:18:05 \(1734358685\)
[ 2131.951045] Lustre: DEBUG MARKER: == sanity test 31g: cross directory link================== 14:18:05 (1734358685)
[ 2136.865705] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31h: cross directory link under child========================================================================= 14:18:11 \(1734358691\)
[ 2137.198987] Lustre: DEBUG MARKER: == sanity test 31h: cross directory link under child========================================================================= 14:18:11 (1734358691)
[ 2138.977491] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31i: cross directory link under parent========================================================================= 14:18:13 \(1734358693\)
[ 2139.306876] Lustre: DEBUG MARKER: == sanity test 31i: cross directory link under parent========================================================================= 14:18:13 (1734358693)
[ 2140.997066] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31j: link for directory =================== 14:18:15 \(1734358695\)
[ 2141.336975] Lustre: DEBUG MARKER: == sanity test 31j: link for directory =================== 14:18:15 (1734358695)
[ 2143.044397] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31k: link to file: the same, non-existing, dir ========================================================== 14:18:17 \(1734358697\)
[ 2143.380734] Lustre: DEBUG MARKER: == sanity test 31k: link to file: the same, non-existing, dir ========================================================== 14:18:17 (1734358697)
[ 2145.469504] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31l: link to file: target dir has trailing slash ========================================================== 14:18:19 \(1734358699\)
[ 2145.810554] Lustre: DEBUG MARKER: == sanity test 31l: link to file: target dir has trailing slash ========================================================== 14:18:19 (1734358699)
[ 2146.207352] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_31l need coreutils \< 8.31 or kernel \>= 5.18 for ln
[ 2146.543722] Lustre: DEBUG MARKER: SKIP: sanity test_31l need coreutils < 8.31 or kernel >= 5.18 for ln
[ 2146.949680] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31m: link to file: the same, non-existing, dir ========================================================== 14:18:21 \(1734358701\)
[ 2147.286682] Lustre: DEBUG MARKER: == sanity test 31m: link to file: the same, non-existing, dir ========================================================== 14:18:21 (1734358701)
[ 2148.965273] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31n: check link count of unlinked file ==== 14:18:23 \(1734358703\)
[ 2149.303735] Lustre: DEBUG MARKER: == sanity test 31n: check link count of unlinked file ==== 14:18:23 (1734358703)
[ 2150.962683] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31o: duplicate hard links with same filename ========================================================== 14:18:25 \(1734358705\)
[ 2151.301872] Lustre: DEBUG MARKER: == sanity test 31o: duplicate hard links with same filename ========================================================== 14:18:25 (1734358705)
[ 2161.401080] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31p: remove of open striped directory ===== 14:18:35 \(1734358715\)
[ 2161.739575] Lustre: DEBUG MARKER: == sanity test 31p: remove of open striped directory ===== 14:18:35 (1734358715)
[ 2162.125261] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_31p needs \>= 2 MDTs
[ 2162.456995] Lustre: DEBUG MARKER: SKIP: sanity test_31p needs >= 2 MDTs
[ 2162.868248] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31q: create striped directory on specific MDTs ========================================================== 14:18:37 \(1734358717\)
[ 2163.213174] Lustre: DEBUG MARKER: == sanity test 31q: create striped directory on specific MDTs ========================================================== 14:18:37 (1734358717)
[ 2163.590174] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_31q needs \>= 3 MDTs
[ 2163.924815] Lustre: DEBUG MARKER: SKIP: sanity test_31q needs >= 3 MDTs
[ 2164.336317] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31r: open-rename\(replace\) race ============ 14:18:38 \(1734358718\)
[ 2164.676089] Lustre: DEBUG MARKER: == sanity test 31r: open-rename(replace) race ============ 14:18:38 (1734358718)
[ 2167.366696] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32a: stat d32a\/ext2-mountpoint\/.. =============================================================================== 14:18:41 \(1734358721\)
[ 2167.695738] Lustre: DEBUG MARKER: == sanity test 32a: stat d32a/ext2-mountpoint/.. =============================================================================== 14:18:41 (1734358721)
[ 2167.842976] loop0: detected capacity change from 0 to 16000
[ 2167.844576] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2167.911197] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2169.536757] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32b: open d32b\/ext2-mountpoint\/.. =============================================================================== 14:18:43 \(1734358723\)
[ 2169.887826] Lustre: DEBUG MARKER: == sanity test 32b: open d32b/ext2-mountpoint/.. =============================================================================== 14:18:43 (1734358723)
[ 2170.041838] loop0: detected capacity change from 0 to 16000
[ 2170.043314] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2170.084955] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2171.721001] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32c: stat d32c\/ext2-mountpoint\/..\/d2\/test_dir =================================================================== 14:18:45 \(1734358725\)
[ 2172.062954] Lustre: DEBUG MARKER: == sanity test 32c: stat d32c/ext2-mountpoint/../d2/test_dir =================================================================== 14:18:45 (1734358725)
[ 2172.212966] loop0: detected capacity change from 0 to 16000
[ 2172.214376] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2172.293086] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2174.001416] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32d: open d32d\/ext2-mountpoint\/..\/d2\/test_dir ========================================================== 14:18:48 \(1734358728\)
[ 2174.329426] Lustre: DEBUG MARKER: == sanity test 32d: open d32d/ext2-mountpoint/../d2/test_dir ========================================================== 14:18:48 (1734358728)
[ 2174.476541] loop0: detected capacity change from 0 to 16000
[ 2174.477886] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2174.538570] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2176.221082] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32e: stat d32e\/symlink-\>tmp\/symlink-\>lustre-subdir ========================================================== 14:18:50 \(1734358730\)
[ 2176.550469] Lustre: DEBUG MARKER: == sanity test 32e: stat d32e/symlink->tmp/symlink->lustre-subdir ========================================================== 14:18:50 (1734358730)
[ 2178.216071] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32f: open d32f\/symlink-\>tmp\/symlink-\>lustre-subdir ========================================================== 14:18:52 \(1734358732\)
[ 2178.550572] Lustre: DEBUG MARKER: == sanity test 32f: open d32f/symlink->tmp/symlink->lustre-subdir ========================================================== 14:18:52 (1734358732)
[ 2180.209762] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32g: stat d32g\/symlink-\>tmp\/symlink-\>lustre-subdir\/2 ========================================================== 14:18:54 \(1734358734\)
[ 2180.539217] Lustre: DEBUG MARKER: == sanity test 32g: stat d32g/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 14:18:54 (1734358734)
[ 2182.232829] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32h: open d32h\/symlink-\>tmp\/symlink-\>lustre-subdir\/2 ========================================================== 14:18:56 \(1734358736\)
[ 2182.565343] Lustre: DEBUG MARKER: == sanity test 32h: open d32h/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 14:18:56 (1734358736)
[ 2184.242470] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32i: stat d32i\/ext2-mountpoint\/..\/test_file ===================================================================== 14:18:58 \(1734358738\)
[ 2184.571284] Lustre: DEBUG MARKER: == sanity test 32i: stat d32i/ext2-mountpoint/../test_file ===================================================================== 14:18:58 (1734358738)
[ 2184.721503] loop0: detected capacity change from 0 to 16000
[ 2184.722906] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2184.817622] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2186.522497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32j: open d32j\/ext2-mountpoint\/..\/test_file ===================================================================== 14:19:00 \(1734358740\)
[ 2186.859302] Lustre: DEBUG MARKER: == sanity test 32j: open d32j/ext2-mountpoint/../test_file ===================================================================== 14:19:00 (1734358740)
[ 2187.010677] loop0: detected capacity change from 0 to 16000
[ 2187.012113] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2187.052603] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2194.031157] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32k: stat d32k\/ext2-mountpoint\/..\/d2\/test_file ================================================================== 14:19:08 \(1734358748\)
[ 2194.367722] Lustre: DEBUG MARKER: == sanity test 32k: stat d32k/ext2-mountpoint/../d2/test_file ================================================================== 14:19:08 (1734358748)
[ 2194.518773] loop0: detected capacity change from 0 to 16000
[ 2194.520140] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2194.550301] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2196.276266] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32l: open d32l\/ext2-mountpoint\/..\/d2\/test_file ================================================================== 14:19:10 \(1734358750\)
[ 2196.607163] Lustre: DEBUG MARKER: == sanity test 32l: open d32l/ext2-mountpoint/../d2/test_file ================================================================== 14:19:10 (1734358750)
[ 2196.756257] loop0: detected capacity change from 0 to 16000
[ 2196.757600] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2196.778711] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2198.601370] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32m: stat d32m\/symlink-\>tmp\/symlink-\>lustre-root ================================================================ 14:19:12 \(1734358752\)
[ 2198.946320] Lustre: DEBUG MARKER: == sanity test 32m: stat d32m/symlink->tmp/symlink->lustre-root ================================================================ 14:19:12 (1734358752)
[ 2200.629659] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32n: open d32n\/symlink-\>tmp\/symlink-\>lustre-root ================================================================ 14:19:14 \(1734358754\)
[ 2200.971531] Lustre: DEBUG MARKER: == sanity test 32n: open d32n/symlink->tmp/symlink->lustre-root ================================================================ 14:19:14 (1734358754)
[ 2202.613990] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32o: stat d32o\/symlink-\>tmp\/symlink-\>lustre-root\/ ========================================================== 14:19:16 \(1734358756\)
[ 2202.942896] Lustre: DEBUG MARKER: == sanity test 32o: stat d32o/symlink->tmp/symlink->lustre-root/ ========================================================== 14:19:16 (1734358756)
[ 2204.634978] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32p: open d32p\/symlink-\>tmp\/symlink-\>lustre-root\/ ========================================================== 14:19:18 \(1734358758\)
[ 2204.967004] Lustre: DEBUG MARKER: == sanity test 32p: open d32p/symlink->tmp/symlink->lustre-root/ ========================================================== 14:19:18 (1734358758)
[ 2205.338388] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_1
[ 2205.672579] Lustre: DEBUG MARKER: 32p_1
[ 2206.060558] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_2
[ 2206.394535] Lustre: DEBUG MARKER: 32p_2
[ 2206.772951] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_3
[ 2207.107688] Lustre: DEBUG MARKER: 32p_3
[ 2207.497719] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_4
[ 2207.830303] Lustre: DEBUG MARKER: 32p_4
[ 2208.229981] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_5
[ 2208.566999] Lustre: DEBUG MARKER: 32p_5
[ 2208.950078] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_6
[ 2209.285161] Lustre: DEBUG MARKER: 32p_6
[ 2209.668895] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_7
[ 2209.997467] Lustre: DEBUG MARKER: 32p_7
[ 2210.394244] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_8
[ 2210.722699] Lustre: DEBUG MARKER: 32p_8
[ 2211.111282] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_9
[ 2211.447605] Lustre: DEBUG MARKER: 32p_9
[ 2211.834779] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_10
[ 2212.168797] Lustre: DEBUG MARKER: 32p_10
[ 2213.832689] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32q: stat follows mountpoints in Lustre \(should return error\) ========================================================== 14:19:27 \(1734358767\)
[ 2214.172824] Lustre: DEBUG MARKER: == sanity test 32q: stat follows mountpoints in Lustre (should return error) ========================================================== 14:19:27 (1734358767)
[ 2214.335890] loop0: detected capacity change from 0 to 16000
[ 2214.337283] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2214.399743] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2216.141938] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32r: opendir follows mountpoints in Lustre \(should return error\) ========================================================== 14:19:30 \(1734358770\)
[ 2216.480711] Lustre: DEBUG MARKER: == sanity test 32r: opendir follows mountpoints in Lustre (should return error) ========================================================== 14:19:30 (1734358770)
[ 2216.646799] loop0: detected capacity change from 0 to 16000
[ 2216.648182] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2216.674883] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2218.375008] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33aa: write file with mode 444 \(should return error\) ========================================================== 14:19:32 \(1734358772\)
[ 2218.701154] Lustre: DEBUG MARKER: == sanity test 33aa: write file with mode 444 (should return error) ========================================================== 14:19:32 (1734358772)
[ 2219.114408] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 33_1
[ 2219.453353] Lustre: DEBUG MARKER: 33_1
[ 2219.837314] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 33_2
[ 2220.172099] Lustre: DEBUG MARKER: 33_2
[ 2221.845586] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33a: test open file\(mode=0444\) with O_RDWR \(should return error\) ========================================================== 14:19:35 \(1734358775\)
[ 2222.182014] Lustre: DEBUG MARKER: == sanity test 33a: test open file(mode=0444) with O_RDWR (should return error) ========================================================== 14:19:35 (1734358775)
[ 2223.878245] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33b: test open file with malformed flags \(No panic\) ========================================================== 14:19:38 \(1734358778\)
[ 2224.227801] Lustre: DEBUG MARKER: == sanity test 33b: test open file with malformed flags (No panic) ========================================================== 14:19:38 (1734358778)
[ 2225.966497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33c: test write_bytes stats =============== 14:19:40 \(1734358780\)
[ 2226.296093] Lustre: DEBUG MARKER: == sanity test 33c: test write_bytes stats =============== 14:19:40 (1734358780)
[ 2228.621999] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33d: openfile with 444 modes and malformed flags under remote dir ========================================================== 14:19:42 \(1734358782\)
[ 2228.964576] Lustre: DEBUG MARKER: == sanity test 33d: openfile with 444 modes and malformed flags under remote dir ========================================================== 14:19:42 (1734358782)
[ 2229.348489] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_33d needs \>= 2 MDTs
[ 2229.687741] Lustre: DEBUG MARKER: SKIP: sanity test_33d needs >= 2 MDTs
[ 2230.096361] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33e: mkdir and striped directory should have same mode ========================================================== 14:19:44 \(1734358784\)
[ 2230.437317] Lustre: DEBUG MARKER: == sanity test 33e: mkdir and striped directory should have same mode ========================================================== 14:19:44 (1734358784)
[ 2230.840685] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_33e needs \>= 2 MDTs
[ 2231.206325] Lustre: DEBUG MARKER: SKIP: sanity test_33e needs >= 2 MDTs
[ 2231.640124] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33f: nonroot user can create, access, and remove a striped directory ========================================================== 14:19:45 \(1734358785\)
[ 2232.027462] Lustre: DEBUG MARKER: == sanity test 33f: nonroot user can create, access, and remove a striped directory ========================================================== 14:19:45 (1734358785)
[ 2232.431072] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_33f needs \>= 2 MDTs
[ 2232.817719] Lustre: DEBUG MARKER: SKIP: sanity test_33f needs >= 2 MDTs
[ 2233.266566] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33g: nonroot user create already existing root created file ========================================================== 14:19:47 \(1734358787\)
[ 2233.626490] Lustre: DEBUG MARKER: == sanity test 33g: nonroot user create already existing root created file ========================================================== 14:19:47 (1734358787)
[ 2235.352405] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33h: temp file is located on the same MDT as target \(crush\) ========================================================== 14:19:49 \(1734358789\)
[ 2235.693212] Lustre: DEBUG MARKER: == sanity test 33h: temp file is located on the same MDT as target (crush) ========================================================== 14:19:49 (1734358789)
[ 2236.097612] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_33h needs \>= 2 MDTs
[ 2236.435415] Lustre: DEBUG MARKER: SKIP: sanity test_33h needs >= 2 MDTs
[ 2236.865123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33hh: temp file is located on the same MDT as target \(crush2\) ========================================================== 14:19:51 \(1734358791\)
[ 2237.254409] Lustre: DEBUG MARKER: == sanity test 33hh: temp file is located on the same MDT as target (crush2) ========================================================== 14:19:51 (1734358791)
[ 2237.664519] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_33hh needs \>= 2 MDTs
[ 2238.039954] Lustre: DEBUG MARKER: SKIP: sanity test_33hh needs >= 2 MDTs
[ 2238.473578] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33i: striped directory can be accessed when one MDT is down ========================================================== 14:19:52 \(1734358792\)
[ 2238.852765] Lustre: DEBUG MARKER: == sanity test 33i: striped directory can be accessed when one MDT is down ========================================================== 14:19:52 (1734358792)
[ 2239.231431] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_33i needs \>= 2 MDTs
[ 2239.585252] Lustre: DEBUG MARKER: SKIP: sanity test_33i needs >= 2 MDTs
[ 2240.013936] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33j: lfs setdirstripe -D -i x,y,x should fail ========================================================== 14:19:54 \(1734358794\)
[ 2240.354148] Lustre: DEBUG MARKER: == sanity test 33j: lfs setdirstripe -D -i x,y,x should fail ========================================================== 14:19:54 (1734358794)
[ 2240.740096] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_33j needs \>= 2 MDTs
[ 2241.074894] Lustre: DEBUG MARKER: SKIP: sanity test_33j needs >= 2 MDTs
[ 2241.518880] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34a: truncate file that has not been opened ===================================================================== 14:19:55 \(1734358795\)
[ 2241.857229] Lustre: DEBUG MARKER: == sanity test 34a: truncate file that has not been opened ===================================================================== 14:19:55 (1734358795)
[ 2243.689231] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34b: O_RDONLY opening file doesn\'t create objects =============================================================== 14:19:57 \(1734358797\)
[ 2244.057799] Lustre: DEBUG MARKER: == sanity test 34b: O_RDONLY opening file doesn't create objects =============================================================== 14:19:57 (1734358797)
[ 2245.817816] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34c: O_RDWR opening file-with-size works ======================================================================== 14:19:59 \(1734358799\)
[ 2246.183208] Lustre: DEBUG MARKER: == sanity test 34c: O_RDWR opening file-with-size works ======================================================================== 14:19:59 (1734358799)
[ 2260.653873] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34d: write to sparse file ======================================================================================= 14:20:14 \(1734358814\)
[ 2260.992339] Lustre: DEBUG MARKER: == sanity test 34d: write to sparse file ======================================================================================= 14:20:14 (1734358814)
[ 2262.699885] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34e: create objects, some with size and some without ============================================================ 14:20:16 \(1734358816\)
[ 2263.066654] Lustre: DEBUG MARKER: == sanity test 34e: create objects, some with size and some without ============================================================ 14:20:16 (1734358816)
[ 2264.807276] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34f: read from a file with no objects until EOF ================================================================= 14:20:18 \(1734358818\)
[ 2265.173702] Lustre: DEBUG MARKER: == sanity test 34f: read from a file with no objects until EOF ================================================================= 14:20:18 (1734358818)
[ 2266.953343] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34g: truncate long file ========================================================================================= 14:20:21 \(1734358821\)
[ 2267.291112] Lustre: DEBUG MARKER: == sanity test 34g: truncate long file ========================================================================================= 14:20:21 (1734358821)
[ 2269.071659] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34h: ftruncate file under grouplock should not block ========================================================== 14:20:23 \(1734358823\)
[ 2269.406139] Lustre: DEBUG MARKER: == sanity test 34h: ftruncate file under grouplock should not block ========================================================== 14:20:23 (1734358823)
[ 2275.810123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 35a: exec file with mode 444 \(should return and not leak\) ========================================================== 14:20:29 \(1734358829\)
[ 2276.146345] Lustre: DEBUG MARKER: == sanity test 35a: exec file with mode 444 (should return and not leak) ========================================================== 14:20:29 (1734358829)
[ 2277.939513] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36a: MDS utime check \(mknod, utime\) ======= 14:20:32 \(1734358832\)
[ 2278.306473] Lustre: DEBUG MARKER: == sanity test 36a: MDS utime check (mknod, utime) ======= 14:20:32 (1734358832)
[ 2280.019956] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36b: OST utime check \(open, utime\) ======== 14:20:34 \(1734358834\)
[ 2280.354846] Lustre: DEBUG MARKER: == sanity test 36b: OST utime check (open, utime) ======== 14:20:34 (1734358834)
[ 2281.660361] Autotest: Test running for 35 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2282.232714] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36c: non-root MDS utime check \(mknod, utime\) ========================================================== 14:20:36 \(1734358836\)
[ 2282.597268] Lustre: DEBUG MARKER: == sanity test 36c: non-root MDS utime check (mknod, utime) ========================================================== 14:20:36 (1734358836)
[ 2284.399983] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36d: non-root OST utime check \(open, utime\) ========================================================== 14:20:38 \(1734358838\)
[ 2284.739214] Lustre: DEBUG MARKER: == sanity test 36d: non-root OST utime check (open, utime) ========================================================== 14:20:38 (1734358838)
[ 2286.490423] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36e: utime on non-owned file \(should return error\) ========================================================== 14:20:40 \(1734358840\)
[ 2286.820960] Lustre: DEBUG MARKER: == sanity test 36e: utime on non-owned file (should return error) ========================================================== 14:20:40 (1734358840)
[ 2288.535254] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 14:20:42 \(1734358842\)
[ 2288.898931] Lustre: DEBUG MARKER: == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 14:20:42 (1734358842)
[ 2292.230678] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36g: FMD cache expiry =============================================================================== 14:20:46 \(1734358846\)
[ 2292.570712] Lustre: DEBUG MARKER: == sanity test 36g: FMD cache expiry =============================================================================== 14:20:46 (1734358846)
[ 2337.662554] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 14:21:31 \(1734358891\)
[ 2338.010866] Lustre: DEBUG MARKER: == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 14:21:31 (1734358891)
[ 2341.533926] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36i: change mtime on striped directory ==== 14:21:35 \(1734358895\)
[ 2341.867887] Lustre: DEBUG MARKER: == sanity test 36i: change mtime on striped directory ==== 14:21:35 (1734358895)
[ 2342.294836] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_36i needs \>= 2 MDTs
[ 2342.655601] Lustre: DEBUG MARKER: SKIP: sanity test_36i needs >= 2 MDTs
[ 2343.117955] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 38: open a regular file with O_DIRECTORY should return -ENOTDIR ============================================================= 14:21:37 \(1734358897\)
[ 2343.492810] Lustre: DEBUG MARKER: == sanity test 38: open a regular file with O_DIRECTORY should return -ENOTDIR ============================================================= 14:21:37 (1734358897)
[ 2345.309610] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39a: mtime changed on create ============== 14:21:39 \(1734358899\)
[ 2345.667460] Lustre: DEBUG MARKER: == sanity test 39a: mtime changed on create ============== 14:21:39 (1734358899)
[ 2363.056203] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39b: mtime change on open, link, unlink, rename ================================================================ 14:21:57 \(1734358917\)
[ 2363.243798] Lustre: lustre-OST0002-osc-ffff89618e2ab000: disconnect after 23s idle
[ 2363.245143] Lustre: Skipped 15 previous similar messages
[ 2363.395950] Lustre: DEBUG MARKER: == sanity test 39b: mtime change on open, link, unlink, rename ================================================================ 14:21:57 (1734358917)
[ 2366.757050] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39c: mtime change on rename ===================================================================================== 14:22:00 \(1734358920\)
[ 2367.130395] Lustre: DEBUG MARKER: == sanity test 39c: mtime change on rename ===================================================================================== 14:22:00 (1734358920)
[ 2377.222105] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39d: create, utime, stat ======================================================================================== 14:22:11 \(1734358931\)
[ 2377.558374] Lustre: DEBUG MARKER: == sanity test 39d: create, utime, stat ======================================================================================== 14:22:11 (1734358931)
[ 2379.242591] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39e: create, stat, utime, stat ================================================================================== 14:22:13 \(1734358933\)
[ 2379.577720] Lustre: DEBUG MARKER: == sanity test 39e: create, stat, utime, stat ================================================================================== 14:22:13 (1734358933)
[ 2381.273890] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39f: create, stat, sleep, utime, stat =========================================================================== 14:22:15 \(1734358935\)
[ 2381.611996] Lustre: DEBUG MARKER: == sanity test 39f: create, stat, sleep, utime, stat =========================================================================== 14:22:15 (1734358935)
[ 2385.477348] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39g: write, chmod, stat ========================================================================================= 14:22:19 \(1734358939\)
[ 2385.899693] Lustre: DEBUG MARKER: == sanity test 39g: write, chmod, stat ========================================================================================= 14:22:19 (1734358939)
[ 2391.138629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39h: write, utime within one second, stat ======================================================================= 14:22:25 \(1734358945\)
[ 2391.471290] Lustre: DEBUG MARKER: == sanity test 39h: write, utime within one second, stat ======================================================================= 14:22:25 (1734358945)
[ 2398.455070] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39i: write, rename, stat ======================================================================================== 14:22:32 \(1734358952\)
[ 2398.786340] Lustre: DEBUG MARKER: == sanity test 39i: write, rename, stat ======================================================================================== 14:22:32 (1734358952)
[ 2402.368504] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39j: write, rename, close, stat ================================================================================= 14:22:36 \(1734358956\)
[ 2402.744508] Lustre: DEBUG MARKER: == sanity test 39j: write, rename, close, stat ================================================================================= 14:22:36 (1734358956)
[ 2403.521657] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 debug_mb=150
[ 2414.120007] LustreError: 150073:0:(osc_io.c:901:osc_io_write_start()) cfs_fail_timeout id 412 sleeping for 1000ms
[ 2415.159836] LustreError: 150073:0:(osc_io.c:901:osc_io_write_start()) cfs_fail_timeout id 412 awake
[ 2415.500185] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug_mb=11
[ 2416.303302] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 2418.936201] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39k: write, utime, close, stat ================================================================================== 14:22:53 \(1734358973\)
[ 2419.304693] Lustre: DEBUG MARKER: == sanity test 39k: write, utime, close, stat ================================================================================== 14:22:53 (1734358973)
[ 2430.209623] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39l: directory atime update ===================================================================================== 14:23:04 \(1734358984\)
[ 2430.600075] Lustre: DEBUG MARKER: == sanity test 39l: directory atime update ===================================================================================== 14:23:04 (1734358984)
[ 2439.970693] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39m: test atime and mtime before 1970 ===== 14:23:14 \(1734358994\)
[ 2440.366425] Lustre: DEBUG MARKER: == sanity test 39m: test atime and mtime before 1970 ===== 14:23:14 (1734358994)
[ 2450.678422] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39n: check that O_NOATIME is honored ====== 14:23:24 \(1734359004\)
[ 2451.070668] Lustre: DEBUG MARKER: == sanity test 39n: check that O_NOATIME is honored ====== 14:23:24 (1734359004)
[ 2478.801411] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39o: directory cached attributes updated after create ========================================================== 14:23:52 \(1734359032\)
[ 2479.158856] Lustre: DEBUG MARKER: == sanity test 39o: directory cached attributes updated after create ========================================================== 14:23:52 (1734359032)
[ 2481.099812] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39p: remote directory cached attributes updated after create ================================================================== 14:23:55 \(1734359035\)
[ 2481.463491] Lustre: DEBUG MARKER: == sanity test 39p: remote directory cached attributes updated after create ================================================================== 14:23:55 (1734359035)
[ 2481.857922] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_39p needs \>= 2 MDTs
[ 2482.218278] Lustre: DEBUG MARKER: SKIP: sanity test_39p needs >= 2 MDTs
[ 2482.690280] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39r: lazy atime update on OST ============= 14:23:56 \(1734359036\)
[ 2483.052716] Lustre: DEBUG MARKER: == sanity test 39r: lazy atime update on OST ============= 14:23:56 (1734359036)
[ 2516.152330] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39q: close won\'t zero out atime =========== 14:24:30 \(1734359070\)
[ 2516.489763] Lustre: DEBUG MARKER: == sanity test 39q: close won't zero out atime =========== 14:24:30 (1734359070)
[ 2518.201993] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39s: relatime is supported ================ 14:24:32 \(1734359072\)
[ 2518.537463] Lustre: DEBUG MARKER: == sanity test 39s: relatime is supported ================ 14:24:32 (1734359072)
[ 2518.673491] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2518.684712] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2518.846318] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 2518.958505] Lustre: Unmounted lustre-client
[ 2518.970087] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2518.980830] Lustre: DEBUG MARKER: mount -t lustre -o relatime 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2519.046682] Lustre: Mounted lustre-client
[ 2524.426962] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2524.438580] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2524.589252] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 2524.663315] Lustre: Unmounted lustre-client
[ 2524.672384] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2524.684174] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2524.741560] Lustre: Mounted lustre-client
[ 2526.309189] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39u: stat race ============================ 14:24:40 \(1734359080\)
[ 2526.639190] Lustre: DEBUG MARKER: == sanity test 39u: stat race ============================ 14:24:40 (1734359080)
[ 2529.030131] Lustre: *** cfs_fail_loc=1434, val=5173***
[ 2529.031150] LustreError: 155012:0:(file.c:6211:ll_getattr_dentry()) cfs_race id 1435 sleeping
[ 2530.030094] LustreError: 155014:0:(file.c:1514:ll_merge_attr_nolock()) cfs_race id 1435 sleeping
[ 2534.252181] LustreError: 155012:0:(file.c:6211:ll_getattr_dentry()) cfs_fail_race id 1435 awake: rc=0
[ 2535.276191] LustreError: 155014:0:(file.c:1514:ll_merge_attr_nolock()) cfs_fail_race id 1435 awake: rc=0
[ 2535.278067] LustreError: 155014:0:(file.c:1519:ll_merge_attr_nolock()) cfs_fail_timeout id 1435 sleeping for 1000ms
[ 2536.316184] LustreError: 155014:0:(file.c:1519:ll_merge_attr_nolock()) cfs_fail_timeout id 1435 awake
[ 2536.317951] LustreError: 155014:0:(file.c:6211:ll_getattr_dentry()) cfs_race id 1435 sleeping
[ 2541.420215] LustreError: 155014:0:(file.c:6211:ll_getattr_dentry()) cfs_fail_race id 1435 awake: rc=0
[ 2543.089362] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 40: failed open\(O_TRUNC\) doesn\'t truncate ======================================================================= 14:24:57 \(1734359097\)
[ 2543.441958] Lustre: DEBUG MARKER: == sanity test 40: failed open(O_TRUNC) doesn't truncate ======================================================================= 14:24:57 (1734359097)
[ 2545.143213] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 41: test small file write + fstat =============================================================================== 14:24:59 \(1734359099\)
[ 2545.479954] Lustre: DEBUG MARKER: == sanity test 41: test small file write + fstat =============================================================================== 14:24:59 (1734359099)
[ 2548.136714] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_42a skipping ALWAYS excluded test 42a
[ 2548.472279] Lustre: DEBUG MARKER: SKIP: sanity test_42a skipping ALWAYS excluded test 42a
[ 2548.857772] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_42b skipping ALWAYS excluded test 42b
[ 2549.209847] Lustre: DEBUG MARKER: SKIP: sanity test_42b skipping ALWAYS excluded test 42b
[ 2549.603953] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_42c skipping ALWAYS excluded test 42c
[ 2549.946932] Lustre: DEBUG MARKER: SKIP: sanity test_42c skipping ALWAYS excluded test 42c
[ 2550.360974] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 14:25:04 \(1734359104\)
[ 2550.697598] Lustre: DEBUG MARKER: == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 14:25:04 (1734359104)
[ 2552.146644] Lustre: DEBUG MARKER: sync
[ 2555.774996] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 2556.437186] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 2558.429431] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 14:25:12 \(1734359112\)
[ 2558.773679] Lustre: DEBUG MARKER: == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 14:25:12 (1734359112)
[ 2559.496056] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 2563.310346] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 2564.760879] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 2568.306988] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 2582.558736] Autotest: Test running for 40 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2644.554989] Lustre: DEBUG MARKER: sync
[ 2646.372964] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 2647.155162] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 2649.182643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 14:26:43 \(1734359203\)
[ 2649.526163] Lustre: DEBUG MARKER: == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 14:26:43 (1734359203)
[ 2652.255760] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43a: open\(RDWR\) of file being executed should return -ETXTBSY ========================================================== 14:26:46 \(1734359206\)
[ 2652.573953] Lustre: DEBUG MARKER: == sanity test 43a: open(RDWR) of file being executed should return -ETXTBSY ========================================================== 14:26:46 (1734359206)
[ 2655.254878] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 14:26:49 \(1734359209\)
[ 2655.588839] Lustre: DEBUG MARKER: == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 14:26:49 (1734359209)
[ 2658.306089] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43c: md5sum of copy into lustre =========== 14:26:52 \(1734359212\)
[ 2658.658759] Lustre: DEBUG MARKER: == sanity test 43c: md5sum of copy into lustre =========== 14:26:52 (1734359212)
[ 2660.354710] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44A: zero length read from a sparse stripe ========================================================== 14:26:54 \(1734359214\)
[ 2660.688345] Lustre: DEBUG MARKER: == sanity test 44A: zero length read from a sparse stripe ========================================================== 14:26:54 (1734359214)
[ 2662.369493] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44a: test sparse pwrite ========================================================================================= 14:26:56 \(1734359216\)
[ 2662.692482] Lustre: DEBUG MARKER: == sanity test 44a: test sparse pwrite ========================================================================================= 14:26:56 (1734359216)
[ 2665.189146] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44b: write one byte at offset 0xfffffffe000 ========================================================== 14:26:59 \(1734359219\)
[ 2665.523559] Lustre: DEBUG MARKER: == sanity test 44b: write one byte at offset 0xfffffffe000 ========================================================== 14:26:59 (1734359219)
[ 2667.265943] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44c: write 1 byte at max_object_bytes - 1 offset ========================================================== 14:27:01 \(1734359221\)
[ 2667.559382] Lustre: DEBUG MARKER: == sanity test 44c: write 1 byte at max_object_bytes - 1 offset ========================================================== 14:27:01 (1734359221)
[ 2669.284093] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44d: if write at position fails \(EFBIG\), so should do append ========================================================== 14:27:03 \(1734359223\)
[ 2669.606358] Lustre: DEBUG MARKER: == sanity test 44d: if write at position fails (EFBIG), so should do append ========================================================== 14:27:03 (1734359223)
[ 2671.320122] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44e: write and read maximal stripes ======= 14:27:05 \(1734359225\)
[ 2671.639880] Lustre: DEBUG MARKER: == sanity test 44e: write and read maximal stripes ======= 14:27:05 (1734359225)
[ 2681.541312] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44f: Check fiemap for sparse files ======== 14:27:15 \(1734359235\)
[ 2681.876239] Lustre: DEBUG MARKER: == sanity test 44f: Check fiemap for sparse files ======== 14:27:15 (1734359235)
[ 2883.317161] Autotest: Test running for 45 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3184.137623] Autotest: Test running for 50 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3485.098343] Autotest: Test running for 55 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3756.205735] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 45: osc io page accounting ================ 14:45:10 \(1734360310\)
[ 3756.557320] Lustre: DEBUG MARKER: == sanity test 45: osc io page accounting ================ 14:45:10 (1734360310)
[ 3774.461473] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 46: dirtying a previously written page ========================================================================== 14:45:28 \(1734360328\)
[ 3774.806141] Lustre: DEBUG MARKER: == sanity test 46: dirtying a previously written page ========================================================================== 14:45:28 (1734360328)
[ 3776.763615] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48a: Access renamed working dir \(should return errors\)=========================================================== 14:45:30 \(1734360330\)
[ 3777.106769] Lustre: DEBUG MARKER: == sanity test 48a: Access renamed working dir (should return errors)=========================================================== 14:45:30 (1734360330)
[ 3778.929178] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48b: Access removed working dir \(should return errors\)=========================================================== 14:45:33 \(1734360333\)
[ 3779.267033] Lustre: DEBUG MARKER: == sanity test 48b: Access removed working dir (should return errors)=========================================================== 14:45:33 (1734360333)
[ 3780.991520] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48c: Access removed working subdir \(should return errors\) ========================================================== 14:45:35 \(1734360335\)
[ 3781.330626] Lustre: DEBUG MARKER: == sanity test 48c: Access removed working subdir (should return errors) ========================================================== 14:45:35 (1734360335)
[ 3783.082111] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48d: Access removed parent subdir \(should return errors\) ========================================================== 14:45:37 \(1734360337\)
[ 3783.402593] Lustre: DEBUG MARKER: == sanity test 48d: Access removed parent subdir (should return errors) ========================================================== 14:45:37 (1734360337)
[ 3785.139192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48e: Access to recreated parent subdir \(should return errors\) ========================================================== 14:45:39 \(1734360339\)
[ 3785.496738] Lustre: DEBUG MARKER: == sanity test 48e: Access to recreated parent subdir (should return errors) ========================================================== 14:45:39 (1734360339)
[ 3785.886806] Autotest: Test running for 60 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3787.275532] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48f: non-zero nlink dir unlink won\'t LBUG\(\) ========================================================== 14:45:41 \(1734360341\)
[ 3787.626522] Lustre: DEBUG MARKER: == sanity test 48f: non-zero nlink dir unlink won't LBUG() ========================================================== 14:45:41 (1734360341)
[ 3788.007049] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_48f needs \>= 2 MDTs
[ 3788.350751] Lustre: DEBUG MARKER: SKIP: sanity test_48f needs >= 2 MDTs
[ 3788.775458] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 49: Change max_pages_per_rpc won\'t break osc extent ========================================================== 14:45:42 \(1734360342\)
[ 3789.111741] Lustre: DEBUG MARKER: == sanity test 49: Change max_pages_per_rpc won't break osc extent ========================================================== 14:45:42 (1734360342)
[ 3791.255606] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 50: special situations: \/proc symlinks ========================================================================= 14:45:45 \(1734360345\)
[ 3791.599525] Lustre: DEBUG MARKER: == sanity test 50: special situations: /proc symlinks ========================================================================= 14:45:45 (1734360345)
[ 3793.281801] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51a: special situations: split htree with empty entry ============================================================ 14:45:47 \(1734360347\)
[ 3793.614411] Lustre: DEBUG MARKER: == sanity test 51a: special situations: split htree with empty entry ============================================================ 14:45:47 (1734360347)
[ 3794.325682] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3795.041696] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3797.660011] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51b: exceed 64k subdirectory nlink limit on create, verify unlink ========================================================== 14:45:51 \(1734360351\)
[ 3797.996738] Lustre: DEBUG MARKER: == sanity test 51b: exceed 64k subdirectory nlink limit on create, verify unlink ========================================================== 14:45:51 (1734360351)
[ 3798.738065] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3827.431666] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3829.540122] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3882.436253] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3885.385085] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51d: check LOV round-robin OST object distribution ========================================================== 14:47:19 \(1734360439\)
[ 3885.725008] Lustre: DEBUG MARKER: == sanity test 51d: check LOV round-robin OST object distribution ========================================================== 14:47:19 (1734360439)
[ 3895.690790] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=3
[ 3896.025746] Lustre: DEBUG MARKER: create files with stripecount=3
[ 3896.724209] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3897.988401] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3899.911339] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 302 objects, 101 are index 0
[ 3900.249659] Lustre: DEBUG MARKER: OST0 has 302 objects, 101 are index 0
[ 3900.641765] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 302 objects, 100 are index 0
[ 3900.982847] Lustre: DEBUG MARKER: OST1 has 302 objects, 100 are index 0
[ 3901.376868] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 302 objects, 100 are index 0
[ 3901.713528] Lustre: DEBUG MARKER: OST2 has 302 objects, 100 are index 0
[ 3902.112178] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 302 objects, 100 are index 0
[ 3902.451533] Lustre: DEBUG MARKER: OST3 has 302 objects, 100 are index 0
[ 3902.843437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 302 objects, 100 are index 0
[ 3903.180398] Lustre: DEBUG MARKER: OST4 has 302 objects, 100 are index 0
[ 3903.577173] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 302 objects, 100 are index 0
[ 3903.911929] Lustre: DEBUG MARKER: OST5 has 302 objects, 100 are index 0
[ 3904.311352] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 302 objects, 101 are index 0
[ 3904.643202] Lustre: DEBUG MARKER: OST6 has 302 objects, 101 are index 0
[ 3905.346496] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3906.294642] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3915.482488] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=4
[ 3915.817600] Lustre: DEBUG MARKER: create files with stripecount=4
[ 3916.531198] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3917.813474] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3919.713843] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 404 objects, 102 are index 0
[ 3920.050731] Lustre: DEBUG MARKER: OST0 has 404 objects, 102 are index 0
[ 3920.450283] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 401 objects, 99 are index 0
[ 3920.805141] Lustre: DEBUG MARKER: OST1 has 401 objects, 99 are index 0
[ 3921.181716] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 402 objects, 100 are index 0
[ 3921.512369] Lustre: DEBUG MARKER: OST2 has 402 objects, 100 are index 0
[ 3921.910258] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 401 objects, 99 are index 0
[ 3922.247923] Lustre: DEBUG MARKER: OST3 has 401 objects, 99 are index 0
[ 3922.674076] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 402 objects, 102 are index 0
[ 3922.987391] Lustre: DEBUG MARKER: OST4 has 402 objects, 102 are index 0
[ 3923.385523] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 402 objects, 99 are index 0
[ 3923.719811] Lustre: DEBUG MARKER: OST5 has 402 objects, 99 are index 0
[ 3924.121269] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 402 objects, 101 are index 0
[ 3924.455712] Lustre: DEBUG MARKER: OST6 has 402 objects, 101 are index 0
[ 3925.149011] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3926.100415] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3935.242138] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=5
[ 3935.579275] Lustre: DEBUG MARKER: create files with stripecount=5
[ 3936.286364] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3937.563005] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3939.536702] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 502 objects, 102 are index 0
[ 3939.879645] Lustre: DEBUG MARKER: OST0 has 502 objects, 102 are index 0
[ 3940.278850] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 503 objects, 100 are index 0
[ 3940.617074] Lustre: DEBUG MARKER: OST1 has 503 objects, 100 are index 0
[ 3941.017731] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 503 objects, 100 are index 0
[ 3941.355167] Lustre: DEBUG MARKER: OST2 has 503 objects, 100 are index 0
[ 3941.774375] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 502 objects, 99 are index 0
[ 3942.082261] Lustre: DEBUG MARKER: OST3 has 502 objects, 99 are index 0
[ 3942.477601] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 502 objects, 100 are index 0
[ 3942.808110] Lustre: DEBUG MARKER: OST4 has 502 objects, 100 are index 0
[ 3943.199017] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 501 objects, 100 are index 0
[ 3943.530152] Lustre: DEBUG MARKER: OST5 has 501 objects, 100 are index 0
[ 3943.928053] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 501 objects, 101 are index 0
[ 3944.267756] Lustre: DEBUG MARKER: OST6 has 501 objects, 101 are index 0
[ 3944.958276] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3945.934626] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3955.077592] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=6
[ 3955.425987] Lustre: DEBUG MARKER: create files with stripecount=6
[ 3956.139936] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3957.551131] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3959.478936] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 602 objects, 101 are index 0
[ 3959.815665] Lustre: DEBUG MARKER: OST0 has 602 objects, 101 are index 0
[ 3960.216145] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 602 objects, 100 are index 0
[ 3960.555485] Lustre: DEBUG MARKER: OST1 has 602 objects, 100 are index 0
[ 3960.956696] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 602 objects, 100 are index 0
[ 3961.295077] Lustre: DEBUG MARKER: OST2 has 602 objects, 100 are index 0
[ 3961.687466] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 602 objects, 100 are index 0
[ 3962.030141] Lustre: DEBUG MARKER: OST3 has 602 objects, 100 are index 0
[ 3962.421867] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 602 objects, 100 are index 0
[ 3962.750068] Lustre: DEBUG MARKER: OST4 has 602 objects, 100 are index 0
[ 3963.141763] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 602 objects, 100 are index 0
[ 3963.480048] Lustre: DEBUG MARKER: OST5 has 602 objects, 100 are index 0
[ 3963.872632] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 602 objects, 101 are index 0
[ 3964.209421] Lustre: DEBUG MARKER: OST6 has 602 objects, 101 are index 0
[ 3964.917855] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3965.885828] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3975.032338] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=7
[ 3975.613198] Lustre: DEBUG MARKER: create files with stripecount=7
[ 3976.314428] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3977.608613] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3979.539986] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 702 objects, 100 are index 0
[ 3979.881545] Lustre: DEBUG MARKER: OST0 has 702 objects, 100 are index 0
[ 3980.280618] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 702 objects, 100 are index 0
[ 3980.621119] Lustre: DEBUG MARKER: OST1 has 702 objects, 100 are index 0
[ 3981.020040] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 702 objects, 100 are index 0
[ 3981.364100] Lustre: DEBUG MARKER: OST2 has 702 objects, 100 are index 0
[ 3981.763092] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 702 objects, 100 are index 0
[ 3982.097887] Lustre: DEBUG MARKER: OST3 has 702 objects, 100 are index 0
[ 3982.516326] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 702 objects, 101 are index 0
[ 3982.834591] Lustre: DEBUG MARKER: OST4 has 702 objects, 101 are index 0
[ 3983.234294] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 702 objects, 100 are index 0
[ 3983.569521] Lustre: DEBUG MARKER: OST5 has 702 objects, 100 are index 0
[ 3983.973939] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 702 objects, 101 are index 0
[ 3984.323609] Lustre: DEBUG MARKER: OST6 has 702 objects, 101 are index 0
[ 3985.011088] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3985.986482] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3989.196359] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51e: check file nlink limit =============== 14:49:03 \(1734360543\)
[ 3989.530073] Lustre: DEBUG MARKER: == sanity test 51e: check file nlink limit =============== 14:49:03 (1734360543)
[ 3994.849312] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4050.678629] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4052.377213] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4088.023165] Autotest: Test running for 65 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 4104.682893] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4107.306246] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51f: check many open files limit ========== 14:51:01 \(1734360661\)
[ 4107.642694] Lustre: DEBUG MARKER: == sanity test 51f: check many open files limit ========== 14:51:01 (1734360661)
[ 4108.373422] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4204.958346] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4206.705047] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4249.876153] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4252.543251] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 52a: append-only flag test \(should return errors\) ========================================================== 14:53:26 \(1734360806\)
[ 4252.873215] Lustre: DEBUG MARKER: == sanity test 52a: append-only flag test (should return errors) ========================================================== 14:53:26 (1734360806)
[ 4253.781931] LustreError: 184039:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff8961cf73b000: inode [0x2000013a3:0xad80:0x0] mdc close failed: rc = -1
[ 4255.361525] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 52b: immutable flag test \(should return errors\) ================================================================= 14:53:29 \(1734360809\)
[ 4255.808826] Lustre: DEBUG MARKER: == sanity test 52b: immutable flag test (should return errors) ================================================================= 14:53:29 (1734360809)
[ 4257.618645] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 53: verify that MDS and OSTs agree on pre-creation ============================================================== 14:53:31 \(1734360811\)
[ 4257.946460] Lustre: DEBUG MARKER: == sanity test 53: verify that MDS and OSTs agree on pre-creation ============================================================== 14:53:31 (1734360811)
[ 4267.893579] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54a: unix domain socket test ============== 14:53:42 \(1734360822\)
[ 4268.238680] Lustre: DEBUG MARKER: == sanity test 54a: unix domain socket test ============== 14:53:42 (1734360822)
[ 4269.966501] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54b: char device works in lustre ================================================================================ 14:53:44 \(1734360824\)
[ 4270.304949] Lustre: DEBUG MARKER: == sanity test 54b: char device works in lustre ================================================================================ 14:53:44 (1734360824)
[ 4271.976362] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54c: block device works in lustre =============================================================================== 14:53:46 \(1734360826\)
[ 4272.317416] Lustre: DEBUG MARKER: == sanity test 54c: block device works in lustre =============================================================================== 14:53:46 (1734360826)
[ 4272.611536] loop3: detected capacity change from 0 to 8200
[ 4273.321523] blk_update_request: operation not supported error, dev loop3, sector 8064 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 4273.333059] blk_update_request: operation not supported error, dev loop3, sector 42 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 4273.400763] blk_update_request: operation not supported error, dev loop3, sector 580 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 4273.705156] EXT4-fs (loop3): mounting ext2 file system using the ext4 subsystem
[ 4273.881551] EXT4-fs (loop3): mounted filesystem without journal. Opts: (null). Quota mode: none.
[ 4276.510392] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54d: fifo device works in lustre ================================================================================ 14:53:50 \(1734360830\)
[ 4276.854519] Lustre: DEBUG MARKER: == sanity test 54d: fifo device works in lustre ================================================================================ 14:53:50 (1734360830)
[ 4278.533834] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54e: console\/tty device works in lustre ================================================================================ 14:53:52 \(1734360832\)
[ 4278.871973] Lustre: DEBUG MARKER: == sanity test 54e: console/tty device works in lustre ================================================================================ 14:53:52 (1734360832)
aaaaaa
[ 4280.579638] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 55a: OBD device life cycle unit tests ===== 14:53:54 \(1734360834\)
[ 4280.918175] Lustre: DEBUG MARKER: == sanity test 55a: OBD device life cycle unit tests ===== 14:53:54 (1734360834)
[ 4281.087155] Lustre: OBD: obd_test_setup
[ 4281.087167] Lustre: OBD: obd_name: obd_name, obd_num: 11, obd_uuid: obd_uuid
[ 4281.087963] Lustre: OBD: class_name2dev(): 11, PASS
[ 4281.089192] Lustre: OBD: class_uuid2dev(): 11, PASS
[ 4281.090069] Lustre: OBD: class_name2obd(): 11, PASS
[ 4281.090966] Lustre: OBD: class_uuid2obd(): 11, PASS
[ 4281.099779] Lustre: OBD: obd_test_cleanup
[ 4282.689727] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 55b: Load and unload max OBD devices ====== 14:53:56 \(1734360836\)
[ 4283.033472] Lustre: DEBUG MARKER: == sanity test 55b: Load and unload max OBD devices ====== 14:53:56 (1734360836)
[ 4371.186005] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56a: check \/usr\/bin\/lfs getstripe ========= 14:55:25 \(1734360925\)
[ 4371.522951] Lustre: DEBUG MARKER: == sanity test 56a: check /usr/bin/lfs getstripe ========= 14:55:25 (1734360925)
[ 4375.426104] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56b: check \/usr\/bin\/lfs getdirstripe ====== 14:55:29 \(1734360929\)
[ 4375.757911] Lustre: DEBUG MARKER: == sanity test 56b: check /usr/bin/lfs getdirstripe ====== 14:55:29 (1734360929)
[ 4377.492559] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56bb: check \/usr\/bin\/lfs getdirstripe layout is YAML ========================================================== 14:55:31 \(1734360931\)
[ 4377.831801] Lustre: DEBUG MARKER: == sanity test 56bb: check /usr/bin/lfs getdirstripe layout is YAML ========================================================== 14:55:31 (1734360931)
[ 4383.716577] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56bc: check \'\/usr\/bin\/lfs getdirstripe --yaml\' params are valid ========================================================== 14:55:37 \(1734360937\)
[ 4384.064665] Lustre: DEBUG MARKER: == sanity test 56bc: check '/usr/bin/lfs getdirstripe --yaml' params are valid ========================================================== 14:55:37 (1734360937)
[ 4384.457191] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_56bc need \>= 2 MDTs
[ 4384.796078] Lustre: DEBUG MARKER: SKIP: sanity test_56bc need >= 2 MDTs
[ 4385.203511] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56c: check \'lfs df\' showing device status ========================================================== 14:55:39 \(1734360939\)
[ 4385.558153] Lustre: DEBUG MARKER: == sanity test 56c: check 'lfs df' showing device status ========================================================== 14:55:39 (1734360939)
[ 4388.773812] Autotest: Test running for 70 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 4410.333112] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56d: \'lfs df -v\' prints only configured devices ========================================================== 14:56:04 \(1734360964\)
[ 4410.667349] Lustre: DEBUG MARKER: == sanity test 56d: 'lfs df -v' prints only configured devices ========================================================== 14:56:04 (1734360964)
[ 4412.349461] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56e: \'lfs df\' Handle non LustreFS & multiple LustreFS ========================================================== 14:56:06 \(1734360966\)
[ 4412.692337] Lustre: DEBUG MARKER: == sanity test 56e: 'lfs df' Handle non LustreFS
[ 4414.409244] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56g: check lfs find -name ================= 14:56:08 \(1734360968\)
[ 4414.743322] Lustre: DEBUG MARKER: == sanity test 56g: check lfs find -name ================= 14:56:08 (1734360968)
[ 4416.651686] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56h: check lfs find ! -name =============== 14:56:10 \(1734360970\)
[ 4416.991025] Lustre: DEBUG MARKER: == sanity test 56h: check lfs find ! -name =============== 14:56:10 (1734360970)
[ 4418.684046] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56i: check \'lfs find -ost UUID\' skips directories ========================================================== 14:56:12 \(1734360972\)
[ 4419.019252] Lustre: DEBUG MARKER: == sanity test 56i: check 'lfs find -ost UUID' skips directories ========================================================== 14:56:12 (1734360972)
[ 4420.722888] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ib: check \'lfs find -ost INDEX_RANGE\' command ========================================================== 14:56:14 \(1734360974\)
[ 4421.054556] Lustre: DEBUG MARKER: == sanity test 56ib: check 'lfs find -ost INDEX_RANGE' command ========================================================== 14:56:14 (1734360974)
[ 4422.758690] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56j: check lfs find -type d =============== 14:56:16 \(1734360976\)
[ 4423.099929] Lustre: DEBUG MARKER: == sanity test 56j: check lfs find -type d =============== 14:56:16 (1734360976)
[ 4424.880234] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56k: check lfs find -type f =============== 14:56:19 \(1734360979\)
[ 4425.216340] Lustre: DEBUG MARKER: == sanity test 56k: check lfs find -type f =============== 14:56:19 (1734360979)
[ 4426.895691] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56l: check lfs find -type b =============== 14:56:21 \(1734360981\)
[ 4427.233142] Lustre: DEBUG MARKER: == sanity test 56l: check lfs find -type b =============== 14:56:21 (1734360981)
[ 4428.933199] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56m: check lfs find -type c =============== 14:56:23 \(1734360983\)
[ 4429.278876] Lustre: DEBUG MARKER: == sanity test 56m: check lfs find -type c =============== 14:56:23 (1734360983)
[ 4430.974387] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56n: check lfs find -type l =============== 14:56:25 \(1734360985\)
[ 4431.317357] Lustre: DEBUG MARKER: == sanity test 56n: check lfs find -type l =============== 14:56:25 (1734360985)
[ 4433.016183] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56o: check lfs find -mtime for old files == 14:56:27 \(1734360987\)
[ 4433.357453] Lustre: DEBUG MARKER: == sanity test 56o: check lfs find -mtime for old files == 14:56:27 (1734360987)
[ 4435.735244] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ob: check lfs find -atime -mtime -ctime with units ========================================================== 14:56:29 \(1734360989\)
[ 4436.084247] Lustre: DEBUG MARKER: == sanity test 56ob: check lfs find -atime -mtime -ctime with units ========================================================== 14:56:29 (1734360989)
[ 4440.131317] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56oc: check lfs find -newerXY work ======== 14:56:34 \(1734360994\)
[ 4440.472961] Lustre: DEBUG MARKER: == sanity test 56oc: check lfs find -newerXY work ======== 14:56:34 (1734360994)
[ 4503.987184] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56od: check lfs find -btime with units ==== 14:57:38 \(1734361058\)
[ 4504.325573] Lustre: DEBUG MARKER: == sanity test 56od: check lfs find -btime with units ==== 14:57:38 (1734361058)
[ 4512.101035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56oe: check lfs find with time range ====== 14:57:46 \(1734361066\)
[ 4512.438989] Lustre: DEBUG MARKER: == sanity test 56oe: check lfs find with time range ====== 14:57:46 (1734361066)
[ 4534.901786] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56p: check lfs find -uid and ! -uid ======= 14:58:09 \(1734361089\)
[ 4535.237117] Lustre: DEBUG MARKER: == sanity test 56p: check lfs find -uid and ! -uid ======= 14:58:09 (1734361089)
[ 4537.172416] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56q: check lfs find -gid and ! -gid ======= 14:58:11 \(1734361091\)
[ 4537.511055] Lustre: DEBUG MARKER: == sanity test 56q: check lfs find -gid and ! -gid ======= 14:58:11 (1734361091)
[ 4539.438000] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56r: check lfs find -size works =========== 14:58:13 \(1734361093\)
[ 4539.771056] Lustre: DEBUG MARKER: == sanity test 56r: check lfs find -size works =========== 14:58:13 (1734361093)
[ 4542.201206] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ra: check lfs find -size -lazy works for data on OSTs ========================================================== 14:58:16 \(1734361096\)
[ 4542.539234] Lustre: DEBUG MARKER: == sanity test 56ra: check lfs find -size -lazy works for data on OSTs ========================================================== 14:58:16 (1734361096)
[ 4545.355436] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56rb: check lfs find --size --ost\/--mdt works ========================================================== 14:58:19 \(1734361099\)
[ 4545.689604] Lustre: DEBUG MARKER: == sanity test 56rb: check lfs find --size --ost/--mdt works ========================================================== 14:58:19 (1734361099)
[ 4547.439720] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56rc: check lfs find --mdt-count\/--mdt-hash works ========================================================== 14:58:21 \(1734361101\)
[ 4547.776261] Lustre: DEBUG MARKER: == sanity test 56rc: check lfs find --mdt-count/--mdt-hash works ========================================================== 14:58:21 (1734361101)
[ 4548.162765] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_56rc needs at least 2 MDTs
[ 4548.490656] Lustre: DEBUG MARKER: SKIP: sanity test_56rc needs at least 2 MDTs
[ 4548.898128] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56rd: check lfs find --printf special files ========================================================== 14:58:23 \(1734361103\)
[ 4549.228265] Lustre: DEBUG MARKER: == sanity test 56rd: check lfs find --printf special files ========================================================== 14:58:23 (1734361103)
[ 4550.988402] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56s: check lfs find -stripe-count works === 14:58:25 \(1734361105\)
[ 4551.328942] Lustre: DEBUG MARKER: == sanity test 56s: check lfs find -stripe-count works === 14:58:25 (1734361105)
[ 4553.315987] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56t: check lfs find -stripe-size works ==== 14:58:27 \(1734361107\)
[ 4553.649493] Lustre: DEBUG MARKER: == sanity test 56t: check lfs find -stripe-size works ==== 14:58:27 (1734361107)
[ 4555.828216] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56u: check lfs find -stripe-index works === 14:58:29 \(1734361109\)
[ 4556.164516] Lustre: DEBUG MARKER: == sanity test 56u: check lfs find -stripe-index works === 14:58:29 (1734361109)
[ 4558.185343] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56v: check \'lfs find -m match with lfs getstripe -m\' ========================================================== 14:58:32 \(1734361112\)
[ 4558.521655] Lustre: DEBUG MARKER: == sanity test 56v: check 'lfs find -m match with lfs getstripe -m' ========================================================== 14:58:32 (1734361112)
[ 4560.503496] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wa: check lfs_migrate -c stripe_count works ========================================================== 14:58:34 \(1734361114\)
[ 4560.840458] Lustre: DEBUG MARKER: == sanity test 56wa: check lfs_migrate -c stripe_count works ========================================================== 14:58:34 (1734361114)
[ 4578.659499] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4579.544576] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4643.978641] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wb: check lfs_migrate pool support ====== 14:59:58 \(1734361198\)
[ 4644.313520] Lustre: DEBUG MARKER: == sanity test 56wb: check lfs_migrate pool support ====== 14:59:58 (1734361198)
[ 4646.848292] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 2>/dev/null || echo foo
[ 4648.250575] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool |
[ 4648.250575] grep -e lustre-OST0000_UUID | sort -u | tr '\n' ' '
[ 4649.268375] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool |
[ 4649.268375] grep -e lustre-OST0000_UUID | sort -u | tr '\n' ' '
[ 4655.350690] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 2>/dev/null || echo foo
[ 4657.038724] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wc: check unrecognized options for lfs_migrate are passed through ========================================================== 15:00:11 \(1734361211\)
[ 4657.377390] Lustre: DEBUG MARKER: == sanity test 56wc: check unrecognized options for lfs_migrate are passed through ========================================================== 15:00:11 (1734361211)
[ 4679.723929] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wd: check lfs_migrate --rsync and --no-rsync work ========================================================== 15:00:33 \(1734361233\)
[ 4680.059792] Lustre: DEBUG MARKER: == sanity test 56wd: check lfs_migrate --rsync and --no-rsync work ========================================================== 15:00:33 (1734361233)
[ 4681.851279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56we: check lfs_migrate --non-direct\|-D support ========================================================== 15:00:35 \(1734361235\)
[ 4682.191958] Lustre: DEBUG MARKER: == sanity test 56we: check lfs_migrate --non-direct|-D support ========================================================== 15:00:35 (1734361235)
[ 4684.078384] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56x: lfs migration support ================ 15:00:38 \(1734361238\)
[ 4684.410119] Lustre: DEBUG MARKER: == sanity test 56x: lfs migration support ================ 15:00:38 (1734361238)
[ 4686.565366] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xa: lfs migration --block support ======= 15:00:40 \(1734361240\)
[ 4686.898248] Lustre: DEBUG MARKER: == sanity test 56xa: lfs migration --block support ======= 15:00:40 (1734361240)
[ 4688.865094] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xb: lfs migration hard link support ===== 15:00:42 \(1734361242\)
[ 4689.199054] Lustre: DEBUG MARKER: == sanity test 56xb: lfs migration hard link support ===== 15:00:42 (1734361242)
[ 4689.758214] Autotest: Test running for 75 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 4719.853665] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xc: lfs migration autostripe ============ 15:01:13 \(1734361273\)
[ 4720.193871] Lustre: DEBUG MARKER: == sanity test 56xc: lfs migration autostripe ============ 15:01:13 (1734361273)
[ 4723.544082] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xd: check lfs_migrate --yaml and --copy support ========================================================== 15:01:17 \(1734361277\)
[ 4723.884316] Lustre: DEBUG MARKER: == sanity test 56xd: check lfs_migrate --yaml and --copy support ========================================================== 15:01:17 (1734361277)
[ 4729.462848] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xe: migrate a composite layout file ===== 15:01:23 \(1734361283\)
[ 4729.796914] Lustre: DEBUG MARKER: == sanity test 56xe: migrate a composite layout file ===== 15:01:23 (1734361283)
[ 4738.924851] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xf: FID is not lost during migration of a composite layout file ========================================================== 15:01:33 \(1734361293\)
[ 4739.266236] Lustre: DEBUG MARKER: == sanity test 56xf: FID is not lost during migration of a composite layout file ========================================================== 15:01:33 (1734361293)
[ 4746.125938] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xg: lfs migrate pool support ============ 15:01:40 \(1734361300\)
[ 4746.460205] Lustre: DEBUG MARKER: == sanity test 56xg: lfs migrate pool support ============ 15:01:40 (1734361300)
[ 4748.954249] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_0 2>/dev/null || echo foo
[ 4751.357260] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_0 |
[ 4751.357260] grep -e lustre-OST0000_UUID | sort -u | tr '\n' ' '
[ 4753.739684] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_1 2>/dev/null || echo foo
[ 4756.140049] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_1 |
[ 4756.140049] grep -e lustre-OST0001_UUID | sort -u | tr '\n' ' '
[ 4758.525273] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_2 2>/dev/null || echo foo
[ 4760.925297] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_2 |
[ 4760.925297] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID | sort -u | tr '\n' ' '
[ 4769.911802] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_0 2>/dev/null || echo foo
[ 4775.719139] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_1 2>/dev/null || echo foo
[ 4784.914836] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_2 2>/dev/null || echo foo
[ 4786.594091] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xh: lfs migrate bandwidth limitation support ========================================================== 15:02:20 \(1734361340\)
[ 4786.935657] Lustre: DEBUG MARKER: == sanity test 56xh: lfs migrate bandwidth limitation support ========================================================== 15:02:20 (1734361340)
[ 4815.372899] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xi: lfs migrate stats support =========== 15:02:49 \(1734361369\)
[ 4815.713330] Lustre: DEBUG MARKER: == sanity test 56xi: lfs migrate stats support =========== 15:02:49 (1734361369)
[ 4825.277404] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xj: lfs migrate -b should not cause starvation of threads on OSS ========================================================== 15:02:59 \(1734361379\)
[ 4825.618992] Lustre: DEBUG MARKER: == sanity test 56xj: lfs migrate -b should not cause starvation of threads on OSS ========================================================== 15:02:59 (1734361379)
[ 4827.327147] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4828.054206] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4832.825155] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xk: lfs mirror resync bandwidth limitation support ========================================================== 15:03:06 \(1734361386\)
[ 4833.169297] Lustre: DEBUG MARKER: == sanity test 56xk: lfs mirror resync bandwidth limitation support ========================================================== 15:03:06 (1734361386)
[ 4842.787679] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xl: lfs mirror resync stats support ===== 15:03:16 \(1734361396\)
[ 4843.134901] Lustre: DEBUG MARKER: == sanity test 56xl: lfs mirror resync stats support ===== 15:03:16 (1734361396)
[ 4847.195535] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56y: lfs find -L raid0\|released =========== 15:03:21 \(1734361401\)
[ 4847.539381] Lustre: DEBUG MARKER: == sanity test 56y: lfs find -L raid0|released =========== 15:03:21 (1734361401)
[ 4849.308369] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56z: lfs find should continue after an error ========================================================== 15:03:23 \(1734361403\)
[ 4849.649142] Lustre: DEBUG MARKER: == sanity test 56z: lfs find should continue after an error ========================================================== 15:03:23 (1734361403)
[ 4851.632425] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56aa: lfs find --size under striped dir === 15:03:25 \(1734361405\)
[ 4851.969654] Lustre: DEBUG MARKER: == sanity test 56aa: lfs find --size under striped dir === 15:03:25 (1734361405)
[ 4852.363529] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_56aa needs \>= 2 MDTs
[ 4852.712593] Lustre: DEBUG MARKER: SKIP: sanity test_56aa needs >= 2 MDTs
[ 4853.143279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ab: lfs find --blocks =================== 15:03:27 \(1734361407\)
[ 4853.498298] Lustre: DEBUG MARKER: == sanity test 56ab: lfs find --blocks =================== 15:03:27 (1734361407)
[ 4856.674045] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56aca: check lfs find -perm with octal representation ========================================================== 15:03:30 \(1734361410\)
[ 4857.018207] Lustre: DEBUG MARKER: == sanity test 56aca: check lfs find -perm with octal representation ========================================================== 15:03:30 (1734361410)
[ 4862.001516] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56acb: check lfs find -perm with symbolic representation ========================================================== 15:03:36 \(1734361416\)
[ 4862.342197] Lustre: DEBUG MARKER: == sanity test 56acb: check lfs find -perm with symbolic representation ========================================================== 15:03:36 (1734361416)
[ 4865.060787] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56acc: check parsing error for lfs find -perm ========================================================== 15:03:39 \(1734361419\)
[ 4865.397763] Lustre: DEBUG MARKER: == sanity test 56acc: check parsing error for lfs find -perm ========================================================== 15:03:39 (1734361419)
[ 4867.094308] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ba: test lfs find --component-end, -start, -count, and -flags ========================================================== 15:03:41 \(1734361421\)
[ 4867.427513] Lustre: DEBUG MARKER: == sanity test 56ba: test lfs find --component-end, -start, -count, and -flags ========================================================== 15:03:41 (1734361421)
[ 4870.210291] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ca: check lfs find --mirror-count\|-N and --mirror-state ========================================================== 15:03:44 \(1734361424\)
[ 4870.540934] Lustre: DEBUG MARKER: == sanity test 56ca: check lfs find --mirror-count|-N and --mirror-state ========================================================== 15:03:44 (1734361424)
[ 4872.837778] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56da: test lfs find with long paths ======= 15:03:46 \(1734361426\)
[ 4873.181353] Lustre: DEBUG MARKER: == sanity test 56da: test lfs find with long paths ======= 15:03:46 (1734361426)
[ 4875.563220] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56db: test \'lfs df -m\' only shows MDT devices ========================================================== 15:03:49 \(1734361429\)
[ 4875.901585] Lustre: DEBUG MARKER: == sanity test 56db: test 'lfs df -m' only shows MDT devices ========================================================== 15:03:49 (1734361429)
[ 4877.596639] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56dc: test \'lfs df -o\' only shows OST devices ========================================================== 15:03:51 \(1734361431\)
[ 4877.930739] Lustre: DEBUG MARKER: == sanity test 56dc: test 'lfs df -o' only shows OST devices ========================================================== 15:03:51 (1734361431)
[ 4879.610977] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56dd: test lfs find with mindepth argument ========================================================== 15:03:53 \(1734361433\)
[ 4879.945796] Lustre: DEBUG MARKER: == sanity test 56dd: test lfs find with mindepth argument ========================================================== 15:03:53 (1734361433)
[ 4881.630958] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ea: test lfs find -printf option ======== 15:03:55 \(1734361435\)
[ 4881.965623] Lustre: DEBUG MARKER: == sanity test 56ea: test lfs find -printf option ======== 15:03:55 (1734361435)
[ 4884.460616] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56ea 2>/dev/null || echo foo
[ 4886.866040] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56ea |
[ 4886.866040] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID | sort -u | tr '\n' ' '
[ 4909.306024] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56ea 2>/dev/null || echo foo
[ 4909.610145] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eaa: test lfs find -printf added functions ========================================================== 15:04:23 \(1734361463\)
[ 4910.046506] Lustre: DEBUG MARKER: == sanity test 56eaa: test lfs find -printf added functions ========================================================== 15:04:23 (1734361463)
[ 4921.524139] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eab: test lfs find -ls function ========= 15:04:35 \(1734361475\)
[ 4921.851761] Lustre: DEBUG MARKER: == sanity test 56eab: test lfs find -ls function ========= 15:04:35 (1734361475)
[ 4924.671654] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eb: check lfs getstripe on symlink ====== 15:04:38 \(1734361478\)
[ 4925.003040] Lustre: DEBUG MARKER: == sanity test 56eb: check lfs getstripe on symlink ====== 15:04:38 (1734361478)
[ 4926.772245] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ebb: check \/usr\/bin\/lfs getdirstripe for FIFO file ========================================================== 15:04:40 \(1734361480\)
[ 4927.119511] Lustre: DEBUG MARKER: == sanity test 56ebb: check /usr/bin/lfs getdirstripe for FIFO file ========================================================== 15:04:40 (1734361480)
[ 4928.816116] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ec: check lfs getstripe,setstripe --hex --yaml ========================================================== 15:04:42 \(1734361482\)
[ 4929.150738] Lustre: DEBUG MARKER: == sanity test 56ec: check lfs getstripe,setstripe --hex --yaml ========================================================== 15:04:42 (1734361482)
[ 4930.902830] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ed: verify new YAML format is valid and back-compatible ========================================================== 15:04:45 \(1734361485\)
[ 4931.241084] Lustre: DEBUG MARKER: == sanity test 56ed: verify new YAML format is valid and back-compatible ========================================================== 15:04:45 (1734361485)
[ 4935.126194] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eda: check lfs find --links ============= 15:04:49 \(1734361489\)
[ 4935.459524] Lustre: DEBUG MARKER: == sanity test 56eda: check lfs find --links ============= 15:04:49 (1734361489)
[ 4937.261577] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56edb: check lfs find --links for directory striped on multiple MDTs ========================================================== 15:04:51 \(1734361491\)
[ 4937.599796] Lustre: DEBUG MARKER: == sanity test 56edb: check lfs find --links for directory striped on multiple MDTs ========================================================== 15:04:51 (1734361491)
[ 4937.990959] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_56edb needs \>= 2 MDTs
[ 4938.339026] Lustre: DEBUG MARKER: SKIP: sanity test_56edb needs >= 2 MDTs
[ 4938.762557] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ef: lfs find with multiple paths ======== 15:04:52 \(1734361492\)
[ 4939.097989] Lustre: DEBUG MARKER: == sanity test 56ef: lfs find with multiple paths ======== 15:04:52 (1734361492)
[ 4940.910166] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eg: lfs find -xattr ===================== 15:04:55 \(1734361495\)
[ 4941.257882] Lustre: DEBUG MARKER: == sanity test 56eg: lfs find -xattr ===================== 15:04:55 (1734361495)
[ 4943.112497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eh: check lfs find --skip =============== 15:04:57 \(1734361497\)
[ 4943.460559] Lustre: DEBUG MARKER: == sanity test 56eh: check lfs find --skip =============== 15:04:57 (1734361497)
[ 4945.777702] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ei: test lfs find --printf prints correct projid for special files ========================================================== 15:04:59 \(1734361499\)
[ 4946.116425] Lustre: DEBUG MARKER: == sanity test 56ei: test lfs find --printf prints correct projid for special files ========================================================== 15:04:59 (1734361499)
[ 4947.932752] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ej: lfs migration --non-block copy ====== 15:05:02 \(1734361502\)
[ 4948.270679] Lustre: DEBUG MARKER: == sanity test 56ej: lfs migration --non-block copy ====== 15:05:02 (1734361502)
[ 4950.681927] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 57a: verify MDS filesystem created with large inodes ============================================================ 15:05:04 \(1734361504\)
[ 4951.022454] Lustre: DEBUG MARKER: == sanity test 57a: verify MDS filesystem created with large inodes ============================================================ 15:05:04 (1734361504)
[ 4953.816040] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 57b: default LOV EAs are stored inside large inodes ============================================================= 15:05:07 \(1734361507\)
[ 4954.154835] Lustre: DEBUG MARKER: == sanity test 57b: default LOV EAs are stored inside large inodes ============================================================= 15:05:07 (1734361507)
[ 4960.140318] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 58: verify cross-platform wire constants ======================================================================== 15:05:14 \(1734361514\)
[ 4960.474804] Lustre: DEBUG MARKER: == sanity test 58: verify cross-platform wire constants ======================================================================== 15:05:14 (1734361514)
[ 4962.176447] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 59: verify cancellation of llog records async =================================================================== 15:05:16 \(1734361516\)
[ 4962.520082] Lustre: DEBUG MARKER: == sanity test 59: verify cancellation of llog records async =================================================================== 15:05:16 (1734361516)
[ 4963.220090] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4964.064125] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4965.551797] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4966.294396] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4982.143430] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60a: llog_test run from kernel module and test llog_reader ========================================================== 15:05:36 \(1734361536\)
[ 4982.490253] Lustre: DEBUG MARKER: == sanity test 60a: llog_test run from kernel module and test llog_reader ========================================================== 15:05:36 (1734361536)
[ 4983.242928] Lustre: DEBUG MARKER: /usr/sbin/lctl mark test_60 run 6708 - from kernel mode
[ 4983.586821] Lustre: DEBUG MARKER: test_60 run 6708 - from kernel mode
[ 4990.542473] Autotest: Test running for 80 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5069.429357] Lustre: lustre-MDT0000-mdc-ffff8961cf73b000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 5069.432212] Lustre: Skipped 2 previous similar messages
[ 5069.433602] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[ 5069.441477] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f201d8ac to 0x5d837553f31780ea
[ 5069.447527] LustreError: 4441:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff8961d58756c0 x1818605127656576/t12885744946(12885744946) o101->lustre-MDT0000-mdc-ffff8961cf73b000@10.240.38.186@tcp:12/10 lens 4616/608 e 0 to 0 dl 1734361639 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'lfs.0' uid:0 gid:0
[ 5069.452058] LustreError: 4441:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 1 previous similar message
[ 5069.454130] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[ 5069.929426] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60b: limit repeated messages from CERROR\/CWARN ========================================================== 15:07:04 \(1734361624\)
[ 5070.265283] Lustre: DEBUG MARKER: == sanity test 60b: limit repeated messages from CERROR/CWARN ========================================================== 15:07:04 (1734361624)
[ 5074.025136] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60c: unlink file when mds full ============ 15:07:08 \(1734361628\)
[ 5074.363824] Lustre: DEBUG MARKER: == sanity test 60c: unlink file when mds full ============ 15:07:08 (1734361628)
[ 5075.064862] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5075.571404] Lustre: 4443:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734361613/real 1734361613] req@ffff896229419040 x1818605135659264/t0(0) o400->lustre-MDT0000-mdc-ffff8961cf73b000@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734361629 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 5075.576534] Lustre: 4443:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 5079.667455] Lustre: 4442:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734361618/real 1734361618] req@ffff89622941a3c0 x1818605135660416/t0(0) o400->lustre-MDT0000-mdc-ffff8961cf73b000@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734361634 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 5081.488701] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5082.957111] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5087.449711] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5089.923911] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60d: test printk console message masking == 15:07:24 \(1734361644\)
[ 5090.259465] Lustre: DEBUG MARKER: == sanity test 60d: test printk console message masking == 15:07:24 (1734361644)
[ 5090.393499] Lustre: DEBUG MARKER: test message ID 26558 15790
[ 5092.010485] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60e: no space while new llog is being created ========================================================== 15:07:26 \(1734361646\)
[ 5092.347881] Lustre: DEBUG MARKER: == sanity test 60e: no space while new llog is being created ========================================================== 15:07:26 (1734361646)
[ 5094.670011] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60f: change debug_path works ============== 15:07:28 \(1734361648\)
[ 5095.004516] Lustre: DEBUG MARKER: == sanity test 60f: change debug_path works ============== 15:07:28 (1734361648)
[ 5095.168738] LustreError: dumping log to /tmp/f60f.sanity.1734361649.248873
[ 5097.333306] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60g: transaction abort won\'t cause MDT hung ========================================================== 15:07:31 \(1734361651\)
[ 5097.676538] Lustre: DEBUG MARKER: == sanity test 60g: transaction abort won't cause MDT hung ========================================================== 15:07:31 (1734361651)
[ 5140.357080] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60h: striped directory with missing stripes can be accessed ========================================================== 15:08:14 \(1734361694\)
[ 5140.693909] Lustre: DEBUG MARKER: == sanity test 60h: striped directory with missing stripes can be accessed ========================================================== 15:08:14 (1734361694)
[ 5141.081249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_60h Need at least 2 MDTs
[ 5141.416503] Lustre: DEBUG MARKER: SKIP: sanity test_60h Need at least 2 MDTs
[ 5141.812179] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_60i skipping SLOW test 60i
[ 5142.152439] Lustre: DEBUG MARKER: SKIP: sanity test_60i skipping SLOW test 60i
[ 5142.554979] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60j: llog_reader reports corruptions ====== 15:08:16 \(1734361696\)
[ 5142.891910] Lustre: DEBUG MARKER: == sanity test 60j: llog_reader reports corruptions ====== 15:08:16 (1734361696)
[ 5151.728844] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 61a: mmap\(\) writes don\'t make sync hang ========================================================================== 15:08:25 \(1734361705\)
[ 5152.070575] Lustre: DEBUG MARKER: == sanity test 61a: mmap() writes don't make sync hang ========================================================================== 15:08:25 (1734361705)
[ 5154.480297] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 61b: mmap\(\) of unstriped file is successful ========================================================== 15:08:28 \(1734361708\)
[ 5154.813707] Lustre: DEBUG MARKER: == sanity test 61b: mmap() of unstriped file is successful ========================================================== 15:08:28 (1734361708)
[ 5156.505537] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 63a: Verify oig_wait interruption does not crash ================================================================= 15:08:30 \(1734361710\)
[ 5156.840191] Lustre: DEBUG MARKER: == sanity test 63a: Verify oig_wait interruption does not crash ================================================================= 15:08:30 (1734361710)
[ 5218.140909] Lustre: DEBUG MARKER: sync
[ 5220.161722] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 5220.817073] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 5222.834452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 63b: async write errors should be returned to fsync ============================================================= 15:09:36 \(1734361776\)
[ 5223.172709] Lustre: DEBUG MARKER: == sanity test 63b: async write errors should be returned to fsync ============================================================= 15:09:36 (1734361776)
[ 5223.950860] Lustre: *** cfs_fail_loc=406, val=0***
[ 5223.951841] LustreError: 261191:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0001-osc-ffff8961cf73b000: prep_req failed: rc = -12
[ 5223.953814] LustreError: 261191:0:(osc_cache.c:2129:osc_check_rpcs()) Write request failed with -12
[ 5226.267954] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5227.800517] Lustre: DEBUG MARKER: sync
[ 5229.290485] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 5229.935440] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 5231.921912] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64a: verify filter grant calculations \(in kernel\) =============================================================== 15:09:46 \(1734361786\)
[ 5232.259156] Lustre: DEBUG MARKER: == sanity test 64a: verify filter grant calculations (in kernel) =============================================================== 15:09:46 (1734361786)
[ 5233.028225] Lustre: DEBUG MARKER: sync
[ 5234.507268] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 5235.153561] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 5237.130888] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_64b skipping SLOW test 64b
[ 5237.460169] Lustre: DEBUG MARKER: SKIP: sanity test_64b skipping SLOW test 64b
[ 5237.864135] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64c: verify grant shrink ================== 15:09:51 \(1734361791\)
[ 5238.194252] Lustre: DEBUG MARKER: == sanity test 64c: verify grant shrink ================== 15:09:51 (1734361791)
[ 5238.966143] Lustre: DEBUG MARKER: sync
[ 5240.880629] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 5241.532462] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 5243.574804] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64d: check grant limit exceed ============= 15:09:57 \(1734361797\)
[ 5243.909859] Lustre: DEBUG MARKER: == sanity test 64d: check grant limit exceed ============= 15:09:57 (1734361797)
[ 5291.336579] Autotest: Test running for 85 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5353.008844] Lustre: DEBUG MARKER: sync
[ 5354.369366] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 5355.119033] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 5367.873647] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64e: check grant consumption \(no grant allocation\) ========================================================== 15:12:01 \(1734361921\)
[ 5368.297085] Lustre: DEBUG MARKER: == sanity test 64e: check grant consumption (no grant allocation) ========================================================== 15:12:01 (1734361921)
[ 5368.435292] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5368.447400] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5368.612302] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 5369.115048] Lustre: Unmounted lustre-client
[ 5369.125376] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5369.137539] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 5369.187001] Lustre: Mounted lustre-client
[ 5371.441057] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5371.453098] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5371.615482] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 5371.660200] Lustre: Unmounted lustre-client
[ 5371.668633] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5371.679324] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 5371.731459] Lustre: Mounted lustre-client
[ 5375.183187] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64f: check grant consumption \(with grant allocation\) ========================================================== 15:12:09 \(1734361929\)
[ 5375.534514] Lustre: DEBUG MARKER: == sanity test 64f: check grant consumption (with grant allocation) ========================================================== 15:12:09 (1734361929)
[ 5375.677910] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5375.689554] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5375.834218] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 5375.877828] Lustre: Unmounted lustre-client
[ 5375.887450] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5375.899228] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 5375.948061] Lustre: Mounted lustre-client
[ 5377.022918] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5377.034316] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5377.181086] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 5377.227956] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5377.238619] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 5379.328891] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64g: grant shrink on MDT ================== 15:12:13 \(1734361933\)
[ 5379.668353] Lustre: DEBUG MARKER: == sanity test 64g: grant shrink on MDT ================== 15:12:13 (1734361933)
[ 5398.262851] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64h: grant shrink on read ================= 15:12:32 \(1734361952\)
[ 5398.607925] Lustre: DEBUG MARKER: == sanity test 64h: grant shrink on read ================= 15:12:32 (1734361952)
[ 5399.156831] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param osc.*OST0000*.grant_shrink);
[ 5399.156831] [[ -z "" ]] && param= ||
[ 5399.156831] param=$(grep <<< "$params");
[ 5399.156831] [[ -z $param ]] && param="$params";
[ 5399.156831] while read s; do echo client $s;
[ 5399.156831] done <<< "$param"
[ 5399.176472] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param osc.*OST0000*.grant_shrink_interval);
[ 5399.176472] [[ -z "" ]] && param= ||
[ 5399.176472] param=$(grep <<< "$params");
[ 5399.176472] [[ -z $param ]] && param="$params";
[ 5399.176472] while read s; do echo client $s;
[ 5399.176472] done <<< "$param"
[ 5413.762298] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n osc.lustre-OST0000-osc-ffff8961cf73f000.grant_shrink=1
[ 5413.777385] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n osc.lustre-OST0000-osc-ffff8961cf73f000.grant_shrink_interval=1200
[ 5414.071978] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64i: shrink on reconnect ================== 15:12:48 \(1734361968\)
[ 5414.466240] Lustre: DEBUG MARKER: == sanity test 64i: shrink on reconnect ================== 15:12:48 (1734361968)
[ 5418.357365] Lustre: lustre-OST0000-osc-ffff8961cf73f000: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 5441.090452] Lustre: lustre-OST0000-osc-ffff8961cf73f000: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[ 5441.092321] Lustre: Skipped 1 previous similar message
[ 5445.975345] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5447.128254] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid
[ 5447.463656] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 5447.791465] Lustre: DEBUG MARKER: lctl get_param -n at_max
[ 5448.066215] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid in FULL state after 0 sec
[ 5448.489581] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 5452.623620] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65a: directory with no stripe info ======== 15:13:26 \(1734362006\)
[ 5452.962489] Lustre: DEBUG MARKER: == sanity test 65a: directory with no stripe info ======== 15:13:26 (1734362006)
[ 5454.706895] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65b: directory setstripe -S stripe_size\*2 -i 0 -c 1 ========================================================== 15:13:28 \(1734362008\)
[ 5455.048593] Lustre: DEBUG MARKER: == sanity test 65b: directory setstripe -S stripe_size*2 -i 0 -c 1 ========================================================== 15:13:28 (1734362008)
[ 5456.774778] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65c: directory setstripe -S stripe_size\*4 -i 1 -c 6 ========================================================== 15:13:30 \(1734362010\)
[ 5457.107294] Lustre: DEBUG MARKER: == sanity test 65c: directory setstripe -S stripe_size*4 -i 1 -c 6 ========================================================== 15:13:30 (1734362010)
[ 5458.843293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65d: directory setstripe -S stripe_size -c stripe_count ========================================================== 15:13:32 \(1734362012\)
[ 5459.176573] Lustre: DEBUG MARKER: == sanity test 65d: directory setstripe -S stripe_size -c stripe_count ========================================================== 15:13:32 (1734362012)
[ 5460.948715] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65e: directory setstripe defaults ========= 15:13:35 \(1734362015\)
[ 5461.270302] Lustre: DEBUG MARKER: == sanity test 65e: directory setstripe defaults ========= 15:13:35 (1734362015)
[ 5463.040627] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65f: dir setstripe permission \(should return error\) ============================================================= 15:13:37 \(1734362017\)
[ 5463.380938] Lustre: DEBUG MARKER: == sanity test 65f: dir setstripe permission (should return error) ============================================================= 15:13:37 (1734362017)
[ 5465.119168] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65g: directory setstripe -d =============== 15:13:39 \(1734362019\)
[ 5465.446007] Lustre: DEBUG MARKER: == sanity test 65g: directory setstripe -d =============== 15:13:39 (1734362019)
[ 5467.203575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65h: directory stripe info inherit ============================================================================== 15:13:41 \(1734362021\)
[ 5467.541229] Lustre: DEBUG MARKER: == sanity test 65h: directory stripe info inherit ============================================================================== 15:13:41 (1734362021)
[ 5469.285337] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65i: various tests to set root directory striping ========================================================== 15:13:43 \(1734362023\)
[ 5469.626872] Lustre: DEBUG MARKER: == sanity test 65i: various tests to set root directory striping ========================================================== 15:13:43 (1734362023)
[ 5477.364732] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65j: set default striping on root directory \(bug 6367\)=========================================================== 15:13:51 \(1734362031\)
[ 5477.706762] Lustre: DEBUG MARKER: == sanity test 65j: set default striping on root directory (bug 6367)=========================================================== 15:13:51 (1734362031)
[ 5480.719027] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65k: validate manual striping works properly with deactivated OSCs ========================================================== 15:13:54 \(1734362034\)
[ 5481.062235] Lustre: DEBUG MARKER: == sanity test 65k: validate manual striping works properly with deactivated OSCs ========================================================== 15:13:54 (1734362034)
[ 5485.011797] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5486.467324] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5489.508977] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5490.569689] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5496.776431] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5498.256906] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5501.355949] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5502.396018] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5508.589706] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5510.116927] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5513.167479] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5514.225602] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5524.091127] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5525.609242] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5528.628550] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5529.689929] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5535.867741] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5537.353584] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5540.491865] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5541.555253] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5547.809534] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5549.323707] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5553.006047] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5554.050087] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5560.334308] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5561.839373] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5564.865603] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5565.930630] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5572.700534] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65l: lfs find on -1 stripe dir ================================================================================== 15:15:26 \(1734362126\)
[ 5573.042957] Lustre: DEBUG MARKER: == sanity test 65l: lfs find on -1 stripe dir ================================================================================== 15:15:26 (1734362126)
[ 5574.776170] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65m: normal user can\'t set filesystem default stripe ========================================================== 15:15:28 \(1734362128\)
[ 5575.118657] Lustre: DEBUG MARKER: == sanity test 65m: normal user can't set filesystem default stripe ========================================================== 15:15:28 (1734362128)
[ 5576.830981] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65n: don\'t inherit default layout from root for new subdirectories ========================================================== 15:15:30 \(1734362130\)
[ 5577.173720] Lustre: DEBUG MARKER: == sanity test 65n: don't inherit default layout from root for new subdirectories ========================================================== 15:15:30 (1734362130)
[ 5579.782683] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65n 2>/dev/null || echo foo
[ 5582.194294] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65n |
[ 5582.194294] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID | sort -u | tr '\n' ' '
[ 5582.591210] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_65n needs \>= 2 MDTs
[ 5582.927277] Lustre: DEBUG MARKER: SKIP: sanity test_65n needs >= 2 MDTs
[ 5590.163813] Autotest: Test running for 90 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5601.112444] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65n 2>/dev/null || echo foo
[ 5601.424191] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65o: pool inheritance for mdt component === 15:15:55 \(1734362155\)
[ 5601.844103] Lustre: DEBUG MARKER: == sanity test 65o: pool inheritance for mdt component === 15:15:55 (1734362155)
[ 5604.347333] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65o 2>/dev/null || echo foo
[ 5606.745533] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65o |
[ 5606.745533] grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID | sort -u | tr '\n' ' '
[ 5628.170606] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65o 2>/dev/null || echo foo
[ 5628.471356] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65p: setstripe with yaml file and huge number ========================================================== 15:16:22 \(1734362182\)
[ 5628.894750] Lustre: DEBUG MARKER: == sanity test 65p: setstripe with yaml file and huge number ========================================================== 15:16:22 (1734362182)
[ 5630.646255] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65q: setstripe with \>=8E offset should fail ========================================================== 15:16:24 \(1734362184\)
[ 5630.973360] Lustre: DEBUG MARKER: == sanity test 65q: setstripe with >=8E offset should fail ========================================================== 15:16:24 (1734362184)
[ 5632.678280] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65r: prevent all-zero offsets ============= 15:16:26 \(1734362186\)
[ 5633.016701] Lustre: DEBUG MARKER: == sanity test 65r: prevent all-zero offsets ============= 15:16:26 (1734362186)
[ 5634.776788] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 66: update inode blocks count on client ========================================================================= 15:16:28 \(1734362188\)
[ 5635.114793] Lustre: DEBUG MARKER: == sanity test 66: update inode blocks count on client ========================================================================= 15:16:28 (1734362188)
[ 5639.473726] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 69: verify oa2dentry return -ENOENT doesn\'t LBUG ================================================================ 15:16:33 \(1734362193\)
[ 5639.813869] Lustre: DEBUG MARKER: == sanity test 69: verify oa2dentry return -ENOENT doesn't LBUG ================================================================ 15:16:33 (1734362193)
[ 5644.664901] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 70a: verify health_check, health_write don\'t explode \(on OST\) ========================================================== 15:16:38 \(1734362198\)
[ 5645.002139] Lustre: DEBUG MARKER: == sanity test 70a: verify health_check, health_write don't explode (on OST) ========================================================== 15:16:38 (1734362198)
[ 5651.484046] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_71 skipping SLOW test 71
[ 5651.831368] Lustre: DEBUG MARKER: SKIP: sanity test_71 skipping SLOW test 71
[ 5652.242719] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 72a: Test that remove suid works properly \(bug5695\) ============================================================== 15:16:46 \(1734362206\)
[ 5652.585883] Lustre: DEBUG MARKER: == sanity test 72a: Test that remove suid works properly (bug5695) ============================================================== 15:16:46 (1734362206)
[ 5654.426177] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 72b: Test that we keep mode setting if without file data changed \(bug 24226\) ========================================================== 15:16:48 \(1734362208\)
[ 5654.769116] Lustre: DEBUG MARKER: == sanity test 72b: Test that we keep mode setting if without file data changed (bug 24226) ========================================================== 15:16:48 (1734362208)
[ 5656.626736] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 73: multiple MDC requests \(should not deadlock\) ========================================================== 15:16:50 \(1734362210\)
[ 5656.964620] Lustre: DEBUG MARKER: == sanity test 73: multiple MDC requests (should not deadlock) ========================================================== 15:16:50 (1734362210)
[ 5684.823477] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 74a: ldlm_enqueue freed-export error path, ls \(shouldn\'t LBUG\) ========================================================== 15:17:18 \(1734362238\)
[ 5685.159992] Lustre: DEBUG MARKER: == sanity test 74a: ldlm_enqueue freed-export error path, ls (shouldn't LBUG) ========================================================== 15:17:18 (1734362238)
[ 5687.386765] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 74b: ldlm_enqueue freed-export error path, touch \(shouldn\'t LBUG\) ========================================================== 15:17:21 \(1734362241\)
[ 5687.721416] Lustre: DEBUG MARKER: == sanity test 74b: ldlm_enqueue freed-export error path, touch (shouldn't LBUG) ========================================================== 15:17:21 (1734362241)
[ 5689.701791] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 74c: ldlm_lock_create error path, \(shouldn\'t LBUG\) ========================================================== 15:17:23 \(1734362243\)
[ 5690.032990] Lustre: DEBUG MARKER: == sanity test 74c: ldlm_lock_create error path, (shouldn't LBUG) ========================================================== 15:17:23 (1734362243)
[ 5690.168058] Lustre: *** cfs_fail_loc=319, val=0***
[ 5691.736869] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 76a: confirm clients recycle inodes properly ============================================================== 15:17:25 \(1734362245\)
[ 5692.056590] Lustre: DEBUG MARKER: == sanity test 76a: confirm clients recycle inodes properly ============================================================== 15:17:25 (1734362245)
[ 5697.508790] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 76b: confirm clients recycle directory inodes properly ============================================================== 15:17:31 \(1734362251\)
[ 5697.844901] Lustre: DEBUG MARKER: == sanity test 76b: confirm clients recycle directory inodes properly ============================================================== 15:17:31 (1734362251)
[ 5705.203735] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77a: normal checksum read\/write operation ========================================================== 15:17:39 \(1734362259\)
[ 5705.531227] Lustre: DEBUG MARKER: == sanity test 77a: normal checksum read/write operation ========================================================== 15:17:39 (1734362259)
[ 5707.558498] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77b: checksum error on client write, read ========================================================== 15:17:41 \(1734362261\)
[ 5707.893147] Lustre: DEBUG MARKER: == sanity test 77b: checksum error on client write, read ========================================================== 15:17:41 (1734362261)
[ 5708.128098] Lustre: *** cfs_fail_loc=409, val=0***
[ 5708.180238] LustreError: lustre-OST0006-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed on the client after we checksummed it - likely false positive due to mmap IO (bug 11742): from 10.240.45.50@tcp inode [0x200001b75:0x1ef3:0x0] object 0x3c0000bd0:1073 extent [0-4194303], original client csum ab4af91c (type 20), server csum ab4af91b (type 20), client csum now ab4af91b
[ 5708.185502] LustreError: 4443:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d28aaa40 x1818605153313536/t8589954554(8589954554) o4->lustre-OST0006-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 0 dl 1734362317 ref 3 fl Interpret:RQU/204/0 rc 0/0 job:'dd.0' uid:0 gid:0
[ 5709.617063] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32, rc = 0
[ 5709.948585] Lustre: DEBUG MARKER: set checksum type to crc32, rc = 0
[ 5710.178539] Lustre: *** cfs_fail_loc=408, val=0***
[ 5710.183107] LustreError: lustre-OST0006-osc-ffff8961cf73f000: BAD READ CHECKSUM: from 10.240.45.50@tcp inode [0x200001b75:0x1ef3:0x0] object 0x3c0000bd0:1073 extent [0-4194303], client ff3c9abc/ff3c9abc, server 30dfc260, cksum_type 1
[ 5710.186500] LustreError: 4443:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d28aad80 x1818605153315328/t0(0) o3->lustre-OST0006-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 0 dl 1734362297 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5711.803555] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to adler, rc = 0
[ 5712.140381] Lustre: DEBUG MARKER: set checksum type to adler, rc = 0
[ 5712.368046] Lustre: *** cfs_fail_loc=408, val=0***
[ 5712.375442] LustreError: lustre-OST0006-osc-ffff8961cf73f000: BAD READ CHECKSUM: from 10.240.45.50@tcp inode [0x200001b75:0x1ef3:0x0] object 0x3c0000bd0:1073 extent [0-4194303], client 242b0143/242b0143, server 9fb50231, cksum_type 2
[ 5712.378723] LustreError: 4443:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d28ab740 x1818605153317760/t0(0) o3->lustre-OST0006-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 0 dl 1734362299 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5713.996889] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32c, rc = 0
[ 5714.330875] Lustre: DEBUG MARKER: set checksum type to crc32c, rc = 0
[ 5714.555044] LustreError: lustre-OST0006-osc-ffff8961cf73f000: BAD READ CHECKSUM: from 10.240.45.50@tcp inode [0x200001b75:0x1ef3:0x0] object 0x3c0000bd0:1073 extent [0-4194303], client cc9a4535/cc9a4535, server 88443a98, cksum_type 4
[ 5714.558479] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d34830c0 x1818605153319296/t0(0) o3->lustre-OST0006-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 0 dl 1734362301 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5715.891022] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 5716.234032] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 5716.460268] Lustre: *** cfs_fail_loc=408, val=0***
[ 5716.461251] Lustre: Skipped 1 previous similar message
[ 5718.060607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5718.402517] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5718.625979] LustreError: lustre-OST0006-osc-ffff8961cf73f000: BAD READ CHECKSUM: from 10.240.45.50@tcp inode [0x200001b75:0x1ef3:0x0] object 0x3c0000bd0:1073 extent [0-4194303], client 2b42f80b/2b42f80b, server ab4af91b, cksum_type 20
[ 5718.629423] LustreError: Skipped 1 previous similar message
[ 5718.630418] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d28a9040 x1818605153322752/t0(0) o3->lustre-OST0006-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 0 dl 1734362305 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5718.634859] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 1 previous similar message
[ 5720.199033] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 5720.533918] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 5722.359259] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 5722.695540] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 5722.921945] LustreError: lustre-OST0006-osc-ffff8961cf73f000: BAD READ CHECKSUM: from 10.240.45.50@tcp inode [0x200001b75:0x1ef3:0x0] object 0x3c0000bd0:1073 extent [0-4194303], client dbe3facd/dbe3facd, server 9b8dfb65, cksum_type 80
[ 5722.925321] LustreError: Skipped 1 previous similar message
[ 5724.464220] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5724.807555] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5726.592380] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77c: checksum error on client read with debug ========================================================== 15:18:00 \(1734362280\)
[ 5726.934407] Lustre: DEBUG MARKER: == sanity test 77c: checksum error on client read with debug ========================================================== 15:18:00 (1734362280)
[ 5728.688251] Lustre: *** cfs_fail_loc=408, val=0***
[ 5728.689259] Lustre: Skipped 3 previous similar messages
[ 5728.691049] Lustre: 4442:0:(osc_request.c:2037:dump_all_bulk_pages()) /tmp/lustre-log-checksum_dump-osc-[0x200001b75:0x1ef4:0x0]:[0-1048575]-983bf764-b7e9f874: dumping checksum data
[ 5728.723007] LustreError: dumping log to /tmp/lustre-log.1734362283.4442
[ 5729.023454] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d28aa700 x1818605153331456/t0(0) o3->lustre-OST0000-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 0 dl 1734362316 ref 2 fl Interpret:RMQU/200/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0
[ 5729.028136] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 2 previous similar messages
[ 5738.535384] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77d: checksum error on OST direct write, read ========================================================== 15:18:12 \(1734362292\)
[ 5738.865189] Lustre: DEBUG MARKER: == sanity test 77d: checksum error on OST direct write, read ========================================================== 15:18:12 (1734362292)
[ 5739.456814] LustreError: lustre-OST0000-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed on the client after we checksummed it - likely false positive due to mmap IO (bug 11742): from 10.240.45.50@tcp inode [0x200001b75:0x1ef6:0x0] object 0x2400013a0:1284 extent [0-4194303], original client csum fec05011 (type 20), server csum fec05010 (type 20), client csum now fec05010
[ 5740.954344] LustreError: lustre-OST0000-osc-ffff8961cf73f000: BAD READ CHECKSUM: from 10.240.45.50@tcp inode [0x200001b75:0x1ef6:0x0] object 0x2400013a0:1284 extent [4194304-8388607], client 872950a1/872950a1, server fec05010, cksum_type 20
[ 5740.957864] LustreError: Skipped 1 previous similar message
[ 5743.645130] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77f: repeat checksum error on write \(expect error\) ========================================================== 15:18:17 \(1734362297\)
[ 5744.006142] Lustre: DEBUG MARKER: == sanity test 77f: repeat checksum error on write (expect error) ========================================================== 15:18:17 (1734362297)
[ 5744.394430] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32, rc = 0
[ 5744.732324] Lustre: DEBUG MARKER: set checksum type to crc32, rc = 0
[ 5744.913468] Lustre: *** cfs_fail_loc=409, val=0***
[ 5744.914465] Lustre: Skipped 2 previous similar messages
[ 5746.039144] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [0-4194303], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5746.043664] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d3480d00 x1818605153338624/t8589958331(8589958331) o4->lustre-OST0001-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 0 dl 1734362315 ref 2 fl Interpret:RMQU/200/0 rc 0/0 job:'directio.0' uid:0 gid:0
[ 5746.048260] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 2 previous similar messages
[ 5750.884282] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [4194304-8388607], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5750.888816] LustreError: Skipped 3 previous similar messages
[ 5754.986407] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [0-4194303], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5754.994507] LustreError: Skipped 1 previous similar message
[ 5766.394713] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [0-4194303], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5766.399254] LustreError: Skipped 3 previous similar messages
[ 5779.599988] Lustre: *** cfs_fail_loc=409, val=0***
[ 5779.600970] Lustre: Skipped 29 previous similar messages
[ 5779.602001] LustreError: 4443:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d3482080 x1818605153343744/t8589958345(8589958345) o4->lustre-OST0001-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 0 dl 1734362366 ref 2 fl Interpret:RMQU/200/0 rc 0/0 job:'ptlrpcd_00_01.0' uid:0 gid:0
[ 5779.606799] LustreError: 4443:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 13 previous similar messages
[ 5787.729357] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [4194304-8388607], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5787.733863] LustreError: Skipped 5 previous similar messages
[ 5809.028726] LustreError: 4442:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 5809.833256] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 5810.128418] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to adler, rc = 0
[ 5810.556909] Lustre: DEBUG MARKER: set checksum type to adler, rc = 0
[ 5823.247522] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [0-4194303], original client csum 7d33b9a0 (type 2), server csum 7d33b99f (type 2), client csum now 7d33b9a0
[ 5823.251997] LustreError: Skipped 11 previous similar messages
[ 5848.796431] Lustre: *** cfs_fail_loc=409, val=0***
[ 5848.797417] Lustre: Skipped 43 previous similar messages
[ 5848.798430] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d28aa3c0 x1818605153352704/t8589958367(8589958367) o4->lustre-OST0001-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 0 dl 1734362435 ref 2 fl Interpret:RMQU/200/0 rc 0/0 job:'ptlrpcd_00_00.0' uid:0 gid:0
[ 5848.803098] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 19 previous similar messages
[ 5880.007619] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 5880.305435] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32c, rc = 0
[ 5880.728115] Lustre: DEBUG MARKER: set checksum type to crc32c, rc = 0
[ 5887.366559] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [0-4194303], original client csum de2bf73f (type 4), server csum de2bf73e (type 4), client csum now de2bf73f
[ 5887.371089] LustreError: Skipped 20 previous similar messages
[ 5891.103276] Autotest: Test running for 95 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5945.432852] LustreError: 4442:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 5945.435483] LustreError: 4442:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 5946.925930] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 5947.342663] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 5982.865063] Lustre: *** cfs_fail_loc=409, val=0***
[ 5982.870305] Lustre: Skipped 87 previous similar messages
[ 5982.871360] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11 req@ffff8961d3481380 x1818605153370240/t8589958411(8589958411) o4->lustre-OST0001-osc-ffff8961cf73f000@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 0 dl 1734362552 ref 2 fl Interpret:RMQU/200/0 rc 0/0 job:'ptlrpcd_00_00.0' uid:0 gid:0
[ 5982.876065] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 39 previous similar messages
[ 6013.536469] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 6013.539045] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 6013.890811] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 6014.314730] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 6015.568248] LustreError: lustre-OST0001-osc-ffff8961cf73f000: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.45.50@tcp inode [0x200001b75:0x1ef7:0x0] object 0x280000bd0:5058 extent [0-4194303], original client csum fec05011 (type 20), server csum fec05010 (type 20), client csum now fec05011
[ 6015.572946] LustreError: Skipped 38 previous similar messages
[ 6076.602798] LustreError: 4442:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 6076.605336] LustreError: 4442:0:(osc_request.c:2602:brw_interpret()) Skipped 2 previous similar messages
[ 6076.890119] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 6077.301976] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 6139.732837] LustreError: 4442:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 6144.498847] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 6144.918887] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 6191.985256] Autotest: Test running for 100 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 6209.559501] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff8961cf73f000: too many resent retries for object: 10737421264:5058: rc = -11
[ 6209.562108] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 6209.964615] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 6210.377261] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 6212.068673] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77g: checksum error on OST write, read ==== 15:26:06 \(1734362766\)
[ 6212.402834] Lustre: DEBUG MARKER: == sanity test 77g: checksum error on OST write, read ==== 15:26:06 (1734362766)
[ 6214.777449] LustreError: lustre-OST0000-osc-ffff8961cf73f000: BAD READ CHECKSUM: from 10.240.45.50@tcp inode [0x200001b75:0x1efa:0x0] object 0x2400013a0:1285 extent [0-4194303], client ab4af91b/ab4af91b, server f10cf420, cksum_type 20
[ 6218.029264] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77k: enable\/disable checksum correctly ==== 15:26:12 \(1734362772\)
[ 6218.360420] Lustre: DEBUG MARKER: == sanity test 77k: enable/disable checksum correctly ==== 15:26:12 (1734362772)
[ 6218.864004] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 6218.878751] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6218.890460] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6219.053648] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 6219.108740] Lustre: Unmounted lustre-client
[ 6219.109599] Lustre: Skipped 1 previous similar message
[ 6219.119057] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6219.131016] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 6219.184686] Lustre: Mounted lustre-client
[ 6219.185506] Lustre: Skipped 1 previous similar message
[ 6219.586569] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 6220.603240] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 6221.619626] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 6221.640877] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6221.652194] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6221.802116] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 6221.847902] Lustre: Unmounted lustre-client
[ 6221.857041] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6221.868024] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 6221.919107] Lustre: Mounted lustre-client
[ 6222.316745] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6222.328217] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6222.477098] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 6222.535231] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6222.546061] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,checksum 10.240.38.186@tcp:/lustre /mnt/lustre
[ 6222.631733] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6222.642741] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6222.783411] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 6222.834278] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6222.845087] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,nochecksum 10.240.38.186@tcp:/lustre /mnt/lustre
[ 6222.926944] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6222.938971] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6223.085026] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 6223.126597] Lustre: Unmounted lustre-client
[ 6223.127469] Lustre: Skipped 2 previous similar messages
[ 6223.135983] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6223.146724] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 6223.203467] Lustre: Mounted lustre-client
[ 6223.204313] Lustre: Skipped 2 previous similar messages
[ 6224.879953] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 6225.896320] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 6226.912801] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 6227.205305] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77l: preferred checksum type is remembered after reconnected ========================================================== 15:26:21 \(1734362781\)
[ 6227.552144] Lustre: DEBUG MARKER: == sanity test 77l: preferred checksum type is remembered after reconnected ========================================================== 15:26:21 (1734362781)
[ 6227.972314] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to invalid, rc = 22
[ 6228.314944] Lustre: DEBUG MARKER: set checksum type to invalid, rc = 22
[ 6228.709728] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32, rc = 0
[ 6229.042510] Lustre: DEBUG MARKER: set checksum type to crc32, rc = 0
[ 6229.218642] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6229.245256] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6229.960728] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6230.290442] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6234.708763] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 4 sec
[ 6235.034658] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 4 sec
[ 6235.181625] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6235.210588] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6235.926763] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6236.262475] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6236.630885] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6236.961484] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6237.358717] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to adler, rc = 0
[ 6237.693882] Lustre: DEBUG MARKER: set checksum type to adler, rc = 0
[ 6237.864870] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6237.891868] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6238.608168] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6238.936858] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6249.428826] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 10 sec
[ 6249.847885] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 10 sec
[ 6249.987816] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6250.015714] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6250.738071] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6251.067180] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6251.442565] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6251.768128] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6252.173826] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32c, rc = 0
[ 6252.505153] Lustre: DEBUG MARKER: set checksum type to crc32c, rc = 0
[ 6252.680057] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6252.708285] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6253.429371] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6253.760761] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6265.258569] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 11 sec
[ 6265.665439] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 11 sec
[ 6265.814150] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6265.841417] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6266.548272] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6266.875800] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6267.259909] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6267.583916] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6267.986195] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 6268.328459] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 6268.500539] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6268.527592] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6269.235894] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6269.561681] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6280.049810] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 10 sec
[ 6280.377757] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 10 sec
[ 6280.525143] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6280.552746] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6281.272376] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6281.607041] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6281.978196] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6282.309998] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6282.718136] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 6283.051500] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 6283.223926] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6283.251146] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6283.973808] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6284.295777] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6295.804460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 11 sec
[ 6296.213689] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 11 sec
[ 6296.363957] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6296.393355] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6297.113028] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6297.442845] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6297.826236] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6298.159929] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6298.581647] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 6298.921476] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 6299.094157] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6299.125723] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6299.872452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6300.195004] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6310.690329] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 10 sec
[ 6311.125991] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 10 sec
[ 6311.272842] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6311.301163] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6312.030396] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6312.359813] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6312.745126] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6313.078145] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6313.485976] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 6313.824758] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 6314.002623] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6314.030252] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6314.742400] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6315.073442] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6326.569609] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 11 sec
[ 6326.985962] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in IDLE state after 11 sec
[ 6327.131374] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6327.159373] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6327.872323] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6328.197850] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid 50
[ 6328.576956] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6328.909004] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d465e800.ost_server_uuid in FULL state after 0 sec
[ 6330.702758] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 6331.038378] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 6331.457044] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77m: Verify checksum_speed is correctly read ========================================================== 15:28:05 \(1734362885\)
[ 6331.791957] Lustre: DEBUG MARKER: == sanity test 77m: Verify checksum_speed is correctly read ========================================================== 15:28:05 (1734362885)
[ 6333.501178] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77n: Verify read from a hole inside contiguous blocks with T10PI ========================================================== 15:28:07 \(1734362887\)
[ 6333.855347] Lustre: DEBUG MARKER: == sanity test 77n: Verify read from a hole inside contiguous blocks with T10PI ========================================================== 15:28:07 (1734362887)
[ 6334.834590] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 6335.172080] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 6335.585032] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 6335.937221] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 6336.337268] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 6336.677955] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 6337.075380] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 6337.416885] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 6339.118022] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 6339.464240] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 6339.874203] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77o: Verify checksum_type for server \(mdt and ofd\(obdfilter\)\) ========================================================== 15:28:13 \(1734362893\)
[ 6340.207811] Lustre: DEBUG MARKER: == sanity test 77o: Verify checksum_type for server (mdt and ofd(obdfilter)) ========================================================== 15:28:13 (1734362893)
[ 6343.667518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 78: handle large O_DIRECT writes correctly ====================================================================== 15:28:17 \(1734362897\)
[ 6344.006558] Lustre: DEBUG MARKER: == sanity test 78: handle large O_DIRECT writes correctly ====================================================================== 15:28:17 (1734362897)
[ 6353.734895] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 79: df report consistency check ================================================================================= 15:28:27 \(1734362907\)
[ 6354.071025] Lustre: DEBUG MARKER: == sanity test 79: df report consistency check ================================================================================= 15:28:27 (1734362907)
[ 6370.329483] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 80: Page eviction is equally fast at high offsets too ========================================================== 15:28:44 \(1734362924\)
[ 6370.665853] Lustre: DEBUG MARKER: == sanity test 80: Page eviction is equally fast at high offsets too ========================================================== 15:28:44 (1734362924)
[ 6373.897306] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 81a: OST should retry write when get -ENOSPC ========================================================================= 15:28:48 \(1734362928\)
[ 6374.233469] Lustre: DEBUG MARKER: == sanity test 81a: OST should retry write when get -ENOSPC ========================================================================= 15:28:48 (1734362928)
[ 6376.715462] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 81b: OST should return -ENOSPC when retry still fails ================================================================= 15:28:50 \(1734362930\)
[ 6377.059312] Lustre: DEBUG MARKER: == sanity test 81b: OST should return -ENOSPC when retry still fails ================================================================= 15:28:50 (1734362930)
[ 6380.726200] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 99: cvs strange file\/directory operations ========================================================== 15:28:54 \(1734362934\)
[ 6381.064907] Lustre: DEBUG MARKER: == sanity test 99: cvs strange file/directory operations ========================================================== 15:28:54 (1734362934)
[ 6391.572516] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 100: check local port using privileged port ========================================================== 15:29:05 \(1734362945\)
[ 6391.922747] Lustre: DEBUG MARKER: == sanity test 100: check local port using privileged port ========================================================== 15:29:05 (1734362945)
[ 6393.889895] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101a: check read-ahead for random reads === 15:29:07 \(1734362947\)
[ 6394.208516] Lustre: DEBUG MARKER: == sanity test 101a: check read-ahead for random reads === 15:29:07 (1734362947)
[ 6434.057931] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101b: check stride-io mode read-ahead =========================================================================== 15:29:48 \(1734362988\)
[ 6434.413692] Lustre: DEBUG MARKER: == sanity test 101b: check stride-io mode read-ahead =========================================================================== 15:29:48 (1734362988)
[ 6471.124212] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101c: check stripe_size aligned read-ahead ========================================================== 15:30:25 \(1734363025\)
[ 6471.466334] Lustre: DEBUG MARKER: == sanity test 101c: check stripe_size aligned read-ahead ========================================================== 15:30:25 (1734363025)
[ 6492.932514] Autotest: Test running for 105 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 6499.074723] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101d: file read with and without read-ahead enabled ========================================================== 15:30:53 \(1734363053\)
[ 6499.414829] Lustre: DEBUG MARKER: == sanity test 101d: file read with and without read-ahead enabled ========================================================== 15:30:53 (1734363053)
[ 6554.011158] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101e: check read-ahead for small read\(1k\) for small files\(500k\) ========================================================== 15:31:48 \(1734363108\)
[ 6554.354635] Lustre: DEBUG MARKER: == sanity test 101e: check read-ahead for small read(1k) for small files(500k) ========================================================== 15:31:48 (1734363108)
[ 6569.126624] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101f: check mmap read performance ========= 15:32:03 \(1734363123\)
[ 6569.449589] Lustre: DEBUG MARKER: == sanity test 101f: check mmap read performance ========= 15:32:03 (1734363123)
[ 6569.836360] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_101f no iozone installed
[ 6570.166716] Lustre: DEBUG MARKER: SKIP: sanity test_101f no iozone installed
[ 6570.583771] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101g: Big bulk\(4\/16 MiB\) readahead ======== 15:32:04 \(1734363124\)
[ 6570.921185] Lustre: DEBUG MARKER: == sanity test 101g: Big bulk(4/16 MiB) readahead ======== 15:32:04 (1734363124)
[ 6574.694694] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6574.706365] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6574.856718] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 6574.932760] Lustre: Unmounted lustre-client
[ 6574.941722] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6574.953165] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 6575.004639] Lustre: Mounted lustre-client
[ 6616.196313] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6616.213749] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6616.352699] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 6617.239673] Lustre: Unmounted lustre-client
[ 6617.248387] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6617.259585] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 6617.315852] Lustre: Mounted lustre-client
[ 6619.014086] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101h: Readahead should cover current read window ========================================================== 15:32:53 \(1734363173\)
[ 6619.348652] Lustre: DEBUG MARKER: == sanity test 101h: Readahead should cover current read window ========================================================== 15:32:53 (1734363173)
[ 6623.707005] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101i: allow current readahead to exceed reservation ========================================================== 15:32:57 \(1734363177\)
[ 6624.059361] Lustre: DEBUG MARKER: == sanity test 101i: allow current readahead to exceed reservation ========================================================== 15:32:57 (1734363177)
[ 6632.711037] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101j: A complete read block should be submitted when no RA ========================================================== 15:33:06 \(1734363186\)
[ 6633.045011] Lustre: DEBUG MARKER: == sanity test 101j: A complete read block should be submitted when no RA ========================================================== 15:33:06 (1734363186)
[ 6641.590399] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101m: read ahead for small file and last stripe of the file ========================================================== 15:33:15 \(1734363195\)
[ 6641.922453] Lustre: DEBUG MARKER: == sanity test 101m: read ahead for small file and last stripe of the file ========================================================== 15:33:15 (1734363195)
[ 6642.481270] Lustre: DEBUG MARKER: Test readahead: size=4096 ramax= iosz=1048576
[ 6643.015290] Lustre: DEBUG MARKER: Test readahead: size=16384 ramax= iosz=1048576
[ 6643.528243] Lustre: DEBUG MARKER: Test readahead: size=16385 ramax= iosz=1048576
[ 6643.813927] Lustre: DEBUG MARKER: Test readahead: size=16383 ramax= iosz=1048576
[ 6653.934880] Lustre: DEBUG MARKER: Test readahead: size=1048577 ramax= iosz=2097152
[ 6656.113543] Lustre: DEBUG MARKER: Test readahead: size=1064960 ramax= iosz=2097152
[ 6656.726779] Lustre: DEBUG MARKER: Test readahead: size=6307840 ramax= iosz=7340032
[ 6658.098580] Lustre: DEBUG MARKER: Test readahead: size=7356416 ramax= iosz=8388608
[ 6659.079789] Lustre: DEBUG MARKER: Test readahead: size=14696448 ramax= iosz=15728640
[ 6662.345855] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102a: user xattr test ============================================================================================ 15:33:36 \(1734363216\)
[ 6662.695589] Lustre: DEBUG MARKER: == sanity test 102a: user xattr test ============================================================================================ 15:33:36 (1734363216)
[ 6664.531589] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102b: getfattr\/setfattr for trusted.lov EAs ========================================================== 15:33:38 \(1734363218\)
[ 6664.893828] Lustre: DEBUG MARKER: == sanity test 102b: getfattr/setfattr for trusted.lov EAs ========================================================== 15:33:38 (1734363218)
[ 6667.825619] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102c: non-root getfattr\/setfattr for lustre.lov EAs ===================================================================== 15:33:41 \(1734363221\)
[ 6668.167335] Lustre: DEBUG MARKER: == sanity test 102c: non-root getfattr/setfattr for lustre.lov EAs ===================================================================== 15:33:41 (1734363221)
[ 6669.963266] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102d: tar restore stripe info from tarfile,not keep osts ========================================================== 15:33:44 \(1734363224\)
[ 6670.297232] Lustre: DEBUG MARKER: == sanity test 102d: tar restore stripe info from tarfile,not keep osts ========================================================== 15:33:44 (1734363224)
[ 6677.176249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102f: tar copy files, not keep osts ======= 15:33:51 \(1734363231\)
[ 6677.517144] Lustre: DEBUG MARKER: == sanity test 102f: tar copy files, not keep osts ======= 15:33:51 (1734363231)
[ 6684.417948] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102h: grow xattr from inside inode to external block ========================================================== 15:33:58 \(1734363238\)
[ 6684.767775] Lustre: DEBUG MARKER: == sanity test 102h: grow xattr from inside inode to external block ========================================================== 15:33:58 (1734363238)
[ 6685.188474] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.big on \/mnt\/lustre\/f102h.sanity
[ 6685.533391] Lustre: DEBUG MARKER: save trusted.big on /mnt/lustre/f102h.sanity
[ 6685.934854] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.sml on \/mnt\/lustre\/f102h.sanity
[ 6686.277485] Lustre: DEBUG MARKER: save trusted.sml on /mnt/lustre/f102h.sanity
[ 6686.678236] Lustre: DEBUG MARKER: /usr/sbin/lctl mark grow trusted.sml on \/mnt\/lustre\/f102h.sanity
[ 6687.014675] Lustre: DEBUG MARKER: grow trusted.sml on /mnt/lustre/f102h.sanity
[ 6687.416874] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trusted.big still valid after growing trusted.sml
[ 6687.756181] Lustre: DEBUG MARKER: trusted.big still valid after growing trusted.sml
[ 6689.470199] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102ha: grow xattr from inside inode to external inode ========================================================== 15:34:03 \(1734363243\)
[ 6689.808949] Lustre: DEBUG MARKER: == sanity test 102ha: grow xattr from inside inode to external inode ========================================================== 15:34:03 (1734363243)
[ 6690.599733] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.big on \/mnt\/lustre\/f102ha.sanity
[ 6690.935122] Lustre: DEBUG MARKER: save trusted.big on /mnt/lustre/f102ha.sanity
[ 6691.346147] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.sml on \/mnt\/lustre\/f102ha.sanity
[ 6691.685092] Lustre: DEBUG MARKER: save trusted.sml on /mnt/lustre/f102ha.sanity
[ 6692.082768] Lustre: DEBUG MARKER: /usr/sbin/lctl mark grow trusted.sml on \/mnt\/lustre\/f102ha.sanity
[ 6692.419192] Lustre: DEBUG MARKER: grow trusted.sml on /mnt/lustre/f102ha.sanity
[ 6692.821676] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trusted.big still valid after growing trusted.sml
[ 6693.163153] Lustre: DEBUG MARKER: trusted.big still valid after growing trusted.sml
[ 6693.596652] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.big on \/mnt\/lustre\/f102ha.sanity
[ 6693.959616] Lustre: DEBUG MARKER: save trusted.big on /mnt/lustre/f102ha.sanity
[ 6695.701420] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102i: lgetxattr test on symbolic link ====================================================================== 15:34:09 \(1734363249\)
[ 6696.037420] Lustre: DEBUG MARKER: == sanity test 102i: lgetxattr test on symbolic link ====================================================================== 15:34:09 (1734363249)
[ 6697.798161] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102j: non-root tar restore stripe info from tarfile, not keep osts ============================================================= 15:34:11 \(1734363251\)
[ 6698.144092] Lustre: DEBUG MARKER: == sanity test 102j: non-root tar restore stripe info from tarfile, not keep osts ============================================================= 15:34:11 (1734363251)
[ 6704.694598] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102k: setfattr without parameter of value shouldn\'t cause a crash ========================================================== 15:34:18 \(1734363258\)
[ 6705.043738] Lustre: DEBUG MARKER: == sanity test 102k: setfattr without parameter of value shouldn't cause a crash ========================================================== 15:34:18 (1734363258)
[ 6706.871074] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102l: listxattr size test ============================================================================================ 15:34:20 \(1734363260\)
[ 6707.211766] Lustre: DEBUG MARKER: == sanity test 102l: listxattr size test ============================================================================================ 15:34:20 (1734363260)
[ 6708.954535] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102m: Ensure listxattr fails on small bufffer ================================================================== 15:34:23 \(1734363263\)
[ 6709.295854] Lustre: DEBUG MARKER: == sanity test 102m: Ensure listxattr fails on small bufffer ================================================================== 15:34:23 (1734363263)
[ 6711.034611] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102n: silently ignore setxattr on internal trusted xattrs ========================================================== 15:34:25 \(1734363265\)
[ 6711.367870] Lustre: DEBUG MARKER: == sanity test 102n: silently ignore setxattr on internal trusted xattrs ========================================================== 15:34:25 (1734363265)
[ 6720.145017] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102p: check setxattr\(2\) correctly fails without permission ========================================================== 15:34:34 \(1734363274\)
[ 6720.480230] Lustre: DEBUG MARKER: == sanity test 102p: check setxattr(2) correctly fails without permission ========================================================== 15:34:34 (1734363274)
[ 6722.212155] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102q: flistxattr should not return trusted.link EAs for orphans ========================================================== 15:34:36 \(1734363276\)
[ 6722.546321] Lustre: DEBUG MARKER: == sanity test 102q: flistxattr should not return trusted.link EAs for orphans ========================================================== 15:34:36 (1734363276)
[ 6724.260119] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102r: set EAs with empty values =========== 15:34:38 \(1734363278\)
[ 6724.589409] Lustre: DEBUG MARKER: == sanity test 102r: set EAs with empty values =========== 15:34:38 (1734363278)
[ 6726.403120] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102s: getting nonexistent xattrs should fail ========================================================== 15:34:40 \(1734363280\)
[ 6726.737797] Lustre: DEBUG MARKER: == sanity test 102s: getting nonexistent xattrs should fail ========================================================== 15:34:40 (1734363280)
[ 6726.876693] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[ 6726.876693] [[ -z "" ]] && param= ||
[ 6726.876693] param=$(grep <<< "$params");
[ 6726.876693] [[ -z $param ]] && param="$params";
[ 6726.876693] while read s; do echo client $s;
[ 6726.876693] done <<< "$param"
[ 6726.912040] LustreError: lustre-MDT0000-mdc-ffff8961d563c000: operation mds_getxattr to node 10.240.38.186@tcp failed: rc = -95
[ 6726.961020] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff8961d563c000.xattr_cache=1
[ 6728.601236] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102t: zero length xattr values handled correctly ========================================================== 15:34:42 \(1734363282\)
[ 6728.935763] Lustre: DEBUG MARKER: == sanity test 102t: zero length xattr values handled correctly ========================================================== 15:34:42 (1734363282)
[ 6729.066602] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[ 6729.066602] [[ -z "" ]] && param= ||
[ 6729.066602] param=$(grep <<< "$params");
[ 6729.066602] [[ -z $param ]] && param="$params";
[ 6729.066602] while read s; do echo client $s;
[ 6729.066602] done <<< "$param"
[ 6729.187680] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff8961d563c000.xattr_cache=1
[ 6730.789968] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103a: acl test ============================ 15:34:44 \(1734363284\)
[ 6731.127881] Lustre: DEBUG MARKER: == sanity test 103a: acl test ============================ 15:34:44 (1734363284)
[ 6774.941279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103b: umask lfs setstripe ================= 15:35:29 \(1734363329\)
[ 6775.288822] Lustre: DEBUG MARKER: == sanity test 103b: umask lfs setstripe ================= 15:35:29 (1734363329)
[ 6793.582226] Autotest: Test running for 110 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 6797.671572] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103c: \'cp -rp\' won\'t set empty acl ======== 15:35:51 \(1734363351\)
[ 6798.015838] Lustre: DEBUG MARKER: == sanity test 103c: 'cp -rp' won't set empty acl ======== 15:35:51 (1734363351)
[ 6799.905029] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103e: inheritance of big amount of default ACLs ========================================================== 15:35:54 \(1734363354\)
[ 6800.271039] Lustre: DEBUG MARKER: == sanity test 103e: inheritance of big amount of default ACLs ========================================================== 15:35:54 (1734363354)
[ 6929.557323] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103f: changelog doesn\'t interfere with default ACLs buffers ========================================================== 15:38:03 \(1734363483\)
[ 6929.901921] Lustre: DEBUG MARKER: == sanity test 103f: changelog doesn't interfere with default ACLs buffers ========================================================== 15:38:03 (1734363483)
[ 6934.197506] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104a: lfs df [-ih] [path] test =================================================================================== 15:38:08 \(1734363488\)
[ 6934.531644] Lustre: DEBUG MARKER: == sanity test 104a: lfs df [-ih] [path] test =================================================================================== 15:38:08 (1734363488)
[ 6934.721285] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request
[ 6934.743504] Lustre: lustre-OST0000-osc-ffff8961d563c000: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 6934.753238] LustreError: lustre-OST0000-osc-ffff8961d563c000: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 6934.756031] Lustre: lustre-OST0000-osc-ffff8961d563c000: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[ 6934.764329] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6934.791662] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6935.510114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state \(FULL\|IDLE\) osc.lustre-OST0000-osc-ffff8961d563c000.ost_server_uuid 50
[ 6935.839683] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8961d563c000.ost_server_uuid 50
[ 6936.214193] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961d563c000.ost_server_uuid in FULL state after 0 sec
[ 6936.565012] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961d563c000.ost_server_uuid in FULL state after 0 sec
[ 6938.374795] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104b: runas -u 500 -g 500 lfs check servers test ============================================================================== 15:38:12 \(1734363492\)
[ 6938.708586] Lustre: DEBUG MARKER: == sanity test 104b: runas -u 500 -g 500 lfs check servers test ============================================================================== 15:38:12 (1734363492)
[ 6940.449836] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104c: Verify df vs lfs_df stays same after recordsize change ========================================================== 15:38:14 \(1734363494\)
[ 6940.798527] Lustre: DEBUG MARKER: == sanity test 104c: Verify df vs lfs_df stays same after recordsize change ========================================================== 15:38:14 (1734363494)
[ 6941.195240] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_104c zfs only test
[ 6941.535900] Lustre: DEBUG MARKER: SKIP: sanity test_104c zfs only test
[ 6941.962482] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104d: runas -u 500 -g 500 lctl dl test ==== 15:38:16 \(1734363496\)
[ 6942.325219] Lustre: DEBUG MARKER: == sanity test 104d: runas -u 500 -g 500 lctl dl test ==== 15:38:16 (1734363496)
[ 6944.051768] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105a: flock when mounted without -o flock test ================================================================== 15:38:18 \(1734363498\)
[ 6944.415735] Lustre: DEBUG MARKER: == sanity test 105a: flock when mounted without -o flock test ================================================================== 15:38:18 (1734363498)
[ 6946.204752] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105b: fcntl when mounted without -o flock test ================================================================== 15:38:20 \(1734363500\)
[ 6946.552217] Lustre: DEBUG MARKER: == sanity test 105b: fcntl when mounted without -o flock test ================================================================== 15:38:20 (1734363500)
[ 6948.314640] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105c: lockf when mounted without -o flock test ========================================================== 15:38:22 \(1734363502\)
[ 6948.661438] Lustre: DEBUG MARKER: == sanity test 105c: lockf when mounted without -o flock test ========================================================== 15:38:22 (1734363502)
[ 6950.407020] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105d: flock race \(should not freeze\) ================================================================== 15:38:24 \(1734363504\)
[ 6950.753264] Lustre: DEBUG MARKER: == sanity test 105d: flock race (should not freeze) ================================================================== 15:38:24 (1734363504)
[ 6950.905867] LustreError: 348581:0:(ldlm_flock.c:850:ldlm_flock_completion_ast()) cfs_fail_timeout id 315 sleeping for 10000ms
[ 6960.993016] LustreError: 348581:0:(ldlm_flock.c:850:ldlm_flock_completion_ast()) cfs_fail_timeout id 315 awake
[ 6962.653739] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105e: Two conflicting flocks from same process ========================================================== 15:38:36 \(1734363516\)
[ 6962.990226] Lustre: DEBUG MARKER: == sanity test 105e: Two conflicting flocks from same process ========================================================== 15:38:36 (1734363516)
[ 6981.150482] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105f: Enqueue same range flocks =========== 15:38:55 \(1734363535\)
[ 6981.488101] Lustre: DEBUG MARKER: == sanity test 105f: Enqueue same range flocks =========== 15:38:55 (1734363535)
[ 6984.290687] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105g: ldlm_lock_debug stack test ========== 15:38:58 \(1734363538\)
[ 6984.629863] Lustre: DEBUG MARKER: == sanity test 105g: ldlm_lock_debug stack test ========== 15:38:58 (1734363538)
[ 6984.779111] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n ldlm.namespaces.*.dump_stack_on_error=1
[ 6984.800823] Lustre: *** cfs_fail_loc=32f, val=0***
[ 6984.801749] LustreError: 350013:0:(ldlm_flock.c:801:ldlm_flock_completion_ast()) ### Test ldlm error stack ns: lustre-MDT0000-mdc-ffff8961d563c000 lock: ffff896190e90600/0x425de93e76538d5f lrc: 4/0,1 mode: PW/PW res: [0x200001b78:0xd1d:0x0].0xc rrc: 2 type: FLK pid: 350012 [0->9223372036854775807] flags: 0x0 nid: local remote: 0x5d837553f3432bc0 expref: -99 pid: 350013 timeout: 0
[ 6984.807041] CPU: 1 PID: 350013 Comm: flocks_test Kdump: loaded Tainted: G OE 5.15.0-94-generic #104-Ubuntu
[ 6984.808832] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 6984.809823] Call Trace:
[ 6984.810313] <TASK>
[ 6984.810782] show_stack+0x52/0x5c
[ 6984.811440] dump_stack_lvl+0x4a/0x63
[ 6984.812134] dump_stack+0x10/0x16
[ 6984.812755] ldlm_flock_completion_ast.cold+0x5/0xa [ptlrpc]
[ 6984.814014] ? unlock_res_and_lock+0x1f/0x30 [ptlrpc]
[ 6984.814942] ? ldlm_lock_enqueue+0x20c/0x550 [ptlrpc]
[ 6984.815873] ldlm_cli_enqueue_fini+0x9f9/0xf70 [ptlrpc]
[ 6984.816830] ldlm_cli_enqueue+0x4df/0xb00 [ptlrpc]
[ 6984.817716] ? ldlm_process_flock_lock+0xa50/0xa50 [ptlrpc]
[ 6984.818716] ? mdc_changelog_cdev_finish+0x220/0x220 [mdc]
[ 6984.819713] mdc_enqueue_base+0x691/0x2630 [mdc]
[ 6984.820532] mdc_enqueue+0x18/0x20 [mdc]
[ 6984.821255] lmv_enqueue+0x2a9/0x4e0 [lmv]
[ 6984.822031] ll_file_flock+0x7d1/0x1460 [lustre]
[ 6984.823035] ? ldlm_process_flock_lock+0xa50/0xa50 [ptlrpc]
[ 6984.824065] ? mdc_changelog_cdev_finish+0x220/0x220 [mdc]
[ 6984.825042] do_lock_file_wait+0x52/0x120
[ 6984.825805] ? kmem_cache_alloc+0x1ab/0x2f0
[ 6984.826585] ? fcntl_setlk+0x42/0x2e0
[ 6984.827260] fcntl_setlk+0x127/0x2e0
[ 6984.827927] ? __handle_mm_fault+0x405/0x6f0
[ 6984.828713] do_fcntl+0x4ce/0x5a0
[ 6984.829349] __x64_sys_fcntl+0xa9/0xd0
[ 6984.830042] do_syscall_64+0x5c/0xc0
[ 6984.830728] ? exit_to_user_mode_prepare+0x37/0xb0
[ 6984.831602] ? irqentry_exit_to_user_mode+0x17/0x20
[ 6984.832473] ? irqentry_exit+0x1d/0x30
[ 6984.833169] ? exc_page_fault+0x89/0x170
[ 6984.833900] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[ 6984.834803] RIP: 0033:0x7f26862d1db7
[ 6984.835482] Code: 00 00 00 89 74 24 0c 89 7c 24 08 48 89 14 24 e8 4f bc f7 ff 48 8b 14 24 8b 74 24 0c 41 89 c0 8b 7c 24 08 b8 48 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 19 44 89 c7 89 04 24 e8 96 bc f7 ff 8b 04 24
[ 6984.838482] RSP: 002b:00007f26861b8d60 EFLAGS: 00000293 ORIG_RAX: 0000000000000048
[ 6984.839777] RAX: ffffffffffffffda RBX: 00007ffe89ae7eb0 RCX: 00007f26862d1db7
[ 6984.840988] RDX: 00007ffe89ae7ec0 RSI: 0000000000000007 RDI: 0000000000000003
[ 6984.842191] RBP: 0000000000000003 R08: 0000000000000000 R09: 00007f26861b8d17
[ 6984.843396] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000007
[ 6984.844612] R13: 0000000000000000 R14: 00007f26862517d0 R15: 00007ffe89ae7dc0
[ 6984.845823] </TASK>
[ 6988.129419] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n ldlm.namespaces.*.dump_stack_on_error=0
[ 6988.424840] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 106: attempt exec of dir followed by chown of that dir ========================================================== 15:39:02 \(1734363542\)
[ 6988.773567] Lustre: DEBUG MARKER: == sanity test 106: attempt exec of dir followed by chown of that dir ========================================================== 15:39:02 (1734363542)
[ 6990.497946] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 107: Coredump on SIG ====================== 15:39:04 \(1734363544\)
[ 6990.838156] Lustre: DEBUG MARKER: == sanity test 107: Coredump on SIG ====================== 15:39:04 (1734363544)
[ 6990.990679] Unsafe core_pattern used with fs.suid_dumpable=2.
[ 6990.990679] Pipe handler or fully qualified core dump path required.
[ 6990.990679] Set kernel.core_pattern before fs.suid_dumpable.
[ 6993.660133] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 110: filename length checking ============= 15:39:07 \(1734363547\)
[ 6993.998065] Lustre: DEBUG MARKER: == sanity test 110: filename length checking ============= 15:39:07 (1734363547)
[ 6995.783682] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 116a: stripe QOS: free space balance ============================================================================= 15:39:09 \(1734363549\)
[ 6996.128654] Lustre: DEBUG MARKER: == sanity test 116a: stripe QOS: free space balance ============================================================================= 15:39:09 (1734363549)
[ 7095.140619] Autotest: Test running for 115 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 7269.357859] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 116b: QoS shouldn\'t LBUG if not enough OSTs found on the 2nd pass ========================================================== 15:43:43 \(1734363823\)
[ 7269.792047] Lustre: DEBUG MARKER: == sanity test 116b: QoS shouldn't LBUG if not enough OSTs found on the 2nd pass ========================================================== 15:43:43 (1734363823)
[ 7273.478621] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 117: verify osd extend ==================== 15:43:47 \(1734363827\)
[ 7273.814827] Lustre: DEBUG MARKER: == sanity test 117: verify osd extend ==================== 15:43:47 (1734363827)
[ 7275.626895] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118a: verify O_SYNC works ================= 15:43:49 \(1734363829\)
[ 7275.963745] Lustre: DEBUG MARKER: == sanity test 118a: verify O_SYNC works ================= 15:43:49 (1734363829)
[ 7278.095353] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118b: Reclaim dirty pages on fatal error ==================================================================== 15:43:52 \(1734363832\)
[ 7278.426567] Lustre: DEBUG MARKER: == sanity test 118b: Reclaim dirty pages on fatal error ==================================================================== 15:43:52 (1734363832)
[ 7281.696961] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_118c skipping ALWAYS excluded test 118c
[ 7282.036367] Lustre: DEBUG MARKER: SKIP: sanity test_118c skipping ALWAYS excluded test 118c
[ 7282.442385] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_118d skipping ALWAYS excluded test 118d
[ 7282.782106] Lustre: DEBUG MARKER: SKIP: sanity test_118d skipping ALWAYS excluded test 118d
[ 7283.198956] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118f: Simulate unrecoverable OSC side error ==================================================================== 15:43:57 \(1734363837\)
[ 7283.538116] Lustre: DEBUG MARKER: == sanity test 118f: Simulate unrecoverable OSC side error ==================================================================== 15:43:57 (1734363837)
[ 7283.982822] Lustre: *** cfs_fail_loc=40a, val=0***
[ 7283.983780] Lustre: Skipped 145 previous similar messages
[ 7283.984761] LustreError: 357498:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0004-osc-ffff8961d563c000: prep_req failed: rc = -22
[ 7283.986800] LustreError: 357498:0:(osc_cache.c:2129:osc_check_rpcs()) Write request failed with -22
[ 7285.747909] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118g: Don\'t stay in wait if we got local -ENOMEM ==================================================================== 15:43:59 \(1734363839\)
[ 7286.090142] Lustre: DEBUG MARKER: == sanity test 118g: Don't stay in wait if we got local -ENOMEM ==================================================================== 15:43:59 (1734363839)
[ 7287.142964] LustreError: 357959:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0000-osc-ffff8961d563c000: prep_req failed: rc = -12
[ 7287.145087] LustreError: 357959:0:(osc_cache.c:2129:osc_check_rpcs()) Write request failed with -12
[ 7288.892869] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118h: Verify timeout in handling recoverables errors ==================================================================== 15:44:03 \(1734363843\)
[ 7289.232643] Lustre: DEBUG MARKER: == sanity test 118h: Verify timeout in handling recoverables errors ==================================================================== 15:44:03 (1734363843)
[ 7290.603058] LustreError: lustre-OST0003-osc-ffff8961d563c000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[ 7290.605039] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff89620720f0c0 x1818605194536064/t0(0) o4->lustre-OST0003-osc-ffff8961d563c000@10.240.45.50@tcp:6/4 lens 4584/224 e 0 to 0 dl 1734363879 ref 2 fl Interpret:RMQU/200/0 rc -5/-5 job:'multiop.0' uid:0 gid:0
[ 7290.609489] LustreError: 4442:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 67 previous similar messages
[ 7291.643937] LustreError: lustre-OST0003-osc-ffff8961d563c000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[ 7294.075917] LustreError: lustre-OST0003-osc-ffff8961d563c000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[ 7300.742214] LustreError: lustre-OST0003-osc-ffff8961d563c000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[ 7300.744141] LustreError: Skipped 1 previous similar message
[ 7300.745136] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) lustre-OST0003-osc-ffff8961d563c000: too many resent retries for object: 12884904912:2030: rc = -5
[ 7300.747605] LustreError: 4443:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 7300.749450] Lustre: 4443:0:(llite_lib.c:4122:ll_dirty_page_discard_warn()) lustre: dirty page discard: 10.240.38.186@tcp:/lustre/fid: [0x200001b78:0x1237:0x0]// may get corrupted (rc -5)
[ 7303.024468] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118i: Fix error before timeout in recoverable error ==================================================================== 15:44:17 \(1734363857\)
[ 7303.346063] Lustre: DEBUG MARKER: == sanity test 118i: Fix error before timeout in recoverable error ==================================================================== 15:44:17 (1734363857)
[ 7315.660319] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118j: Simulate unrecoverable OST side error ==================================================================== 15:44:29 \(1734363869\)
[ 7315.978237] Lustre: DEBUG MARKER: == sanity test 118j: Simulate unrecoverable OST side error ==================================================================== 15:44:29 (1734363869)
[ 7316.867827] LustreError: lustre-OST0000-osc-ffff8961d563c000: operation ost_write to node 10.240.45.50@tcp failed: rc = -14
[ 7316.869773] LustreError: Skipped 3 previous similar messages
[ 7318.914228] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118k: bio alloc -ENOMEM and IO TERM handling =================================================================== 15:44:33 \(1734363873\)
[ 7319.264126] Lustre: DEBUG MARKER: == sanity test 118k: bio alloc -ENOMEM and IO TERM handling =================================================================== 15:44:33 (1734363873)
[ 7322.941449] LustreError: 4443:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff89620720e700 x1818605194572160/t0(0) o4->lustre-OST0006-osc-ffff8961d563c000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734363912 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_01.0' uid:0 gid:0
[ 7322.946106] LustreError: 4443:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 28 previous similar messages
[ 7334.401674] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118l: fsync dir =========================== 15:44:48 \(1734363888\)
[ 7334.746232] Lustre: DEBUG MARKER: == sanity test 118l: fsync dir =========================== 15:44:48 (1734363888)
[ 7336.469474] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118m: fdatasync dir ======================= 15:44:50 \(1734363890\)
[ 7336.807359] Lustre: DEBUG MARKER: == sanity test 118m: fdatasync dir ======================= 15:44:50 (1734363890)
[ 7338.527255] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118n: statfs\(\) sends OST_STATFS requests in parallel ========================================================== 15:44:52 \(1734363892\)
[ 7338.865045] Lustre: DEBUG MARKER: == sanity test 118n: statfs() sends OST_STATFS requests in parallel ========================================================== 15:44:52 (1734363892)
[ 7343.475050] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119a: Short directIO read must return actual read amount ========================================================== 15:44:57 \(1734363897\)
[ 7343.816399] Lustre: DEBUG MARKER: == sanity test 119a: Short directIO read must return actual read amount ========================================================== 15:44:57 (1734363897)
[ 7347.199708] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119b: Sparse directIO read must return actual read amount ========================================================== 15:45:01 \(1734363901\)
[ 7347.536612] Lustre: DEBUG MARKER: == sanity test 119b: Sparse directIO read must return actual read amount ========================================================== 15:45:01 (1734363901)
[ 7349.769014] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119c: Testing for direct read hitting hole ========================================================== 15:45:03 \(1734363903\)
[ 7350.107967] Lustre: DEBUG MARKER: == sanity test 119c: Testing for direct read hitting hole ========================================================== 15:45:03 (1734363903)
[ 7352.212765] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119e: Basic tests of dio read and write at various sizes ========================================================== 15:45:06 \(1734363906\)
[ 7352.570184] Lustre: DEBUG MARKER: == sanity test 119e: Basic tests of dio read and write at various sizes ========================================================== 15:45:06 (1734363906)
[ 7396.104014] Autotest: Test running for 120 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 7408.240868] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119f: dio vs dio race ===================== 15:46:02 \(1734363962\)
[ 7408.574599] Lustre: DEBUG MARKER: == sanity test 119f: dio vs dio race ===================== 15:46:02 (1734363962)
[ 7584.208507] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119g: dio vs buffered I\/O race ============ 15:48:58 \(1734364138\)
[ 7584.550044] Lustre: DEBUG MARKER: == sanity test 119g: dio vs buffered I/O race ============ 15:48:58 (1734364138)
[ 7632.508788] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119h: basic tests of memory unaligned dio ========================================================== 15:49:46 \(1734364186\)
[ 7632.853404] Lustre: DEBUG MARKER: == sanity test 119h: basic tests of memory unaligned dio ========================================================== 15:49:46 (1734364186)
[ 7651.425509] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_119i skipping ALWAYS excluded test 119i
[ 7651.766235] Lustre: DEBUG MARKER: SKIP: sanity test_119i skipping ALWAYS excluded test 119i
[ 7652.176801] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119j: basic tests of hybrid IO switching == 15:50:06 \(1734364206\)
[ 7652.515751] Lustre: DEBUG MARKER: == sanity test 119j: basic tests of hybrid IO switching == 15:50:06 (1734364206)
[ 7652.968926] Lustre: *** cfs_fail_loc=1429, val=0***
[ 7655.132349] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119m: Test DIO readv\/writev: exercise iter duplication ========================================================== 15:50:09 \(1734364209\)
[ 7655.482041] Lustre: DEBUG MARKER: == sanity test 119m: Test DIO readv/writev: exercise iter duplication ========================================================== 15:50:09 (1734364209)
[ 7658.452952] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119n: Test Unaligned DIO readv\(\) and writev\(\) with unpatched ZFS ========================================================== 15:50:12 \(1734364212\)
[ 7658.789078] Lustre: DEBUG MARKER: == sanity test 119n: Test Unaligned DIO readv() and writev() with unpatched ZFS ========================================================== 15:50:12 (1734364212)
[ 7659.195316] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_119n need ZFS server without unaligned_dio support
[ 7659.537406] Lustre: DEBUG MARKER: SKIP: sanity test_119n need ZFS server without unaligned_dio support
[ 7659.953314] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119o: Test Unaligned DIO readv\(\) and writev\(\) with unpatched servers ========================================================== 15:50:14 \(1734364214\)
[ 7660.290124] Lustre: DEBUG MARKER: == sanity test 119o: Test Unaligned DIO readv() and writev() with unpatched servers ========================================================== 15:50:14 (1734364214)
[ 7660.694675] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_119o need ldiskfs without unaligned_dio support.
[ 7661.031148] Lustre: DEBUG MARKER: SKIP: sanity test_119o need ldiskfs without unaligned_dio support.
[ 7661.471619] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119p: Test Unaligned DIO readv\(\) and writev\(\) with patched servers ========================================================== 15:50:15 \(1734364215\)
[ 7661.807340] Lustre: DEBUG MARKER: == sanity test 119p: Test Unaligned DIO readv() and writev() with patched servers ========================================================== 15:50:15 (1734364215)
[ 7697.353101] Autotest: Test running for 125 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 7701.368867] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119q: Test patchded Unaligned DIO readv\(\) and writev\(\) ========================================================== 15:50:55 \(1734364255\)
[ 7701.704686] Lustre: DEBUG MARKER: == sanity test 119q: Test patchded Unaligned DIO readv() and writev() ========================================================== 15:50:55 (1734364255)
[ 7752.343090] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120a: Early Lock Cancel: mkdir test ======= 15:51:46 \(1734364306\)
[ 7752.680291] Lustre: DEBUG MARKER: == sanity test 120a: Early Lock Cancel: mkdir test ======= 15:51:46 (1734364306)
[ 7755.259024] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120b: Early Lock Cancel: create test ====== 15:51:49 \(1734364309\)
[ 7755.600258] Lustre: DEBUG MARKER: == sanity test 120b: Early Lock Cancel: create test ====== 15:51:49 (1734364309)
[ 7758.149106] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120c: Early Lock Cancel: link test ======== 15:51:52 \(1734364312\)
[ 7758.497427] Lustre: DEBUG MARKER: == sanity test 120c: Early Lock Cancel: link test ======== 15:51:52 (1734364312)
[ 7761.080998] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120d: Early Lock Cancel: setattr test ===== 15:51:55 \(1734364315\)
[ 7761.415512] Lustre: DEBUG MARKER: == sanity test 120d: Early Lock Cancel: setattr test ===== 15:51:55 (1734364315)
[ 7764.039169] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120e: Early Lock Cancel: unlink test ====== 15:51:58 \(1734364318\)
[ 7764.380726] Lustre: DEBUG MARKER: == sanity test 120e: Early Lock Cancel: unlink test ====== 15:51:58 (1734364318)
[ 7773.991594] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120f: Early Lock Cancel: rename test ====== 15:52:08 \(1734364328\)
[ 7774.337452] Lustre: DEBUG MARKER: == sanity test 120f: Early Lock Cancel: rename test ====== 15:52:08 (1734364328)
[ 7784.764937] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120g: Early Lock Cancel: performance test ========================================================== 15:52:18 \(1734364338\)
[ 7785.105050] Lustre: DEBUG MARKER: == sanity test 120g: Early Lock Cancel: performance test ========================================================== 15:52:18 (1734364338)
[ 7786.265605] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7796.021747] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7844.062739] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 121: read cancel race ===================== 15:53:18 \(1734364398\)
[ 7844.398177] Lustre: DEBUG MARKER: == sanity test 121: read cancel race ===================== 15:53:18 (1734364398)
[ 7845.506931] Lustre: *** cfs_fail_loc=310, val=0***
[ 7847.093996] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123aa: verify statahead work ============== 15:53:21 \(1734364401\)
[ 7847.429558] Lustre: DEBUG MARKER: == sanity test 123aa: verify statahead work ============== 15:53:21 (1734364401)
[ 7849.269420] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files without statahead: 0 sec
[ 7849.609215] Lustre: DEBUG MARKER: ls -l 100 files without statahead: 0 sec
[ 7850.208863] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files with statahead: 0 sec
[ 7850.544393] Lustre: DEBUG MARKER: ls -l 100 files with statahead: 0 sec
[ 7851.257432] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7852.698823] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7856.575647] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files without statahead: 2 sec
[ 7856.919193] Lustre: DEBUG MARKER: ls -l 1000 files without statahead: 2 sec
[ 7858.366950] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files with statahead: 1 sec
[ 7858.706825] Lustre: DEBUG MARKER: ls -l 1000 files with statahead: 1 sec
[ 7859.418138] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7868.212453] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7896.227242] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files without statahead: 26 sec
[ 7896.656703] Lustre: DEBUG MARKER: ls -l 10000 files without statahead: 26 sec
[ 7906.553300] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files with statahead: 8 sec
[ 7906.918682] Lustre: DEBUG MARKER: ls -l 10000 files with statahead: 8 sec
[ 7907.328016] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l done
[ 7907.664815] Lustre: DEBUG MARKER: ls -l done
[ 7926.063238] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123aa.sanity: 18 seconds
[ 7926.480403] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123aa.sanity: 18 seconds
[ 7929.086370] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123ab: verify statahead work by using statx ========================================================== 15:54:43 \(1734364483\)
[ 7929.460572] Lustre: DEBUG MARKER: == sanity test 123ab: verify statahead work by using statx ========================================================== 15:54:43 (1734364483)
[ 7931.392407] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 100 files without statahead: 0 sec
[ 7931.733287] Lustre: DEBUG MARKER: statx -l 100 files without statahead: 0 sec
[ 7932.312671] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 100 files with statahead: 0 sec
[ 7932.643586] Lustre: DEBUG MARKER: statx -l 100 files with statahead: 0 sec
[ 7933.366967] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7934.758258] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7938.520376] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 1000 files without statahead: 2 sec
[ 7938.866697] Lustre: DEBUG MARKER: statx -l 1000 files without statahead: 2 sec
[ 7940.234220] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 1000 files with statahead: 1 sec
[ 7940.573516] Lustre: DEBUG MARKER: statx -l 1000 files with statahead: 1 sec
[ 7941.280731] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7949.860794] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7976.887393] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 10000 files without statahead: 24 sec
[ 7977.359299] Lustre: DEBUG MARKER: statx -l 10000 files without statahead: 24 sec
[ 7986.918576] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 10000 files with statahead: 8 sec
[ 7987.277278] Lustre: DEBUG MARKER: statx -l 10000 files with statahead: 8 sec
[ 7987.678047] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l done
[ 7988.069783] Lustre: DEBUG MARKER: statx -l done
[ 7998.895820] Autotest: Test running for 130 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8006.712216] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ab.sanity: 18 seconds
[ 8007.058020] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ab.sanity: 18 seconds
[ 8009.833626] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123ac: verify statahead work by using statx without glimpse RPCs ========================================================== 15:56:03 \(1734364563\)
[ 8010.176706] Lustre: DEBUG MARKER: == sanity test 123ac: verify statahead work by using statx without glimpse RPCs ========================================================== 15:56:03 (1734364563)
[ 8011.967074] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 100 files without statahead: 1 sec
[ 8012.304347] Lustre: DEBUG MARKER: statx -c %n
[ 8012.837853] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 100 files with statahead: 0 sec
[ 8013.162475] Lustre: DEBUG MARKER: statx -c %n
[ 8013.856475] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8015.355766] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8018.168752] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 1000 files without statahead: 2 sec
[ 8018.505596] Lustre: DEBUG MARKER: statx -c %n
[ 8019.565915] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 1000 files with statahead: 0 sec
[ 8019.909383] Lustre: DEBUG MARKER: statx -c %n
[ 8020.602116] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8029.555591] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8046.659770] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 10000 files without statahead: 15 sec
[ 8047.090074] Lustre: DEBUG MARKER: statx -c %n
[ 8053.298370] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 10000 files with statahead: 5 sec
[ 8053.642193] Lustre: DEBUG MARKER: statx -c %n
[ 8054.356959] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8072.742748] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8121.212811] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 30000 files without statahead: 44 sec
[ 8121.636997] Lustre: DEBUG MARKER: statx -c %n
[ 8139.319964] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 30000 files with statahead: 14 sec
[ 8139.752531] Lustre: DEBUG MARKER: statx -c %n
[ 8140.150628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D done
[ 8140.492466] Lustre: DEBUG MARKER: statx -c %n
[ 8193.438886] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ac.sanity: 53 seconds
[ 8193.877042] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ac.sanity: 53 seconds
[ 8195.899400] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100 files without statahead: 1 sec
[ 8196.231507] Lustre: DEBUG MARKER: statx --cached=always -D 100 files without statahead: 1 sec
[ 8196.663046] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100 files with statahead: 0 sec
[ 8196.997517] Lustre: DEBUG MARKER: statx --cached=always -D 100 files with statahead: 0 sec
[ 8197.706498] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8199.132575] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8200.456695] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 1000 files without statahead: 0 sec
[ 8200.768960] Lustre: DEBUG MARKER: statx --cached=always -D 1000 files without statahead: 0 sec
[ 8201.242149] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 1000 files with statahead: 0 sec
[ 8201.578840] Lustre: DEBUG MARKER: statx --cached=always -D 1000 files with statahead: 0 sec
[ 8202.290616] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8211.240393] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8213.391263] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 10000 files without statahead: 0 sec
[ 8213.729300] Lustre: DEBUG MARKER: statx --cached=always -D 10000 files without statahead: 0 sec
[ 8214.220969] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 10000 files with statahead: 0 sec
[ 8214.561270] Lustre: DEBUG MARKER: statx --cached=always -D 10000 files with statahead: 0 sec
[ 8215.276942] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8296.087636] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8299.912537] Autotest: Test running for 135 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8307.056101] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100000 files without statahead: 1 sec
[ 8307.396964] Lustre: DEBUG MARKER: statx --cached=always -D 100000 files without statahead: 1 sec
[ 8308.474543] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100000 files with statahead: 1 sec
[ 8308.822611] Lustre: DEBUG MARKER: statx --cached=always -D 100000 files with statahead: 1 sec
[ 8309.241198] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D done
[ 8309.582638] Lustre: DEBUG MARKER: statx --cached=always -D done
[ 8600.939794] Autotest: Test running for 140 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8613.946501] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ac.sanity: 304 seconds
[ 8614.369647] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ac.sanity: 304 seconds
[ 8616.928072] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:06:11 \(1734365171\)
[ 8617.270374] Lustre: DEBUG MARKER: == sanity test 123ad: Verify batching statahead works correctly ========================================================== 16:06:11 (1734365171)
[ 8619.113501] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files without statahead: 1 sec
[ 8619.465766] Lustre: DEBUG MARKER: ls -l 100 files without statahead: 1 sec
[ 8620.067190] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files with statahead: 0 sec
[ 8620.408419] Lustre: DEBUG MARKER: ls -l 100 files with statahead: 0 sec
[ 8621.128988] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8622.617588] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8626.502471] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files without statahead: 2 sec
[ 8626.851712] Lustre: DEBUG MARKER: ls -l 1000 files without statahead: 2 sec
[ 8628.691717] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files with statahead: 1 sec
[ 8629.030153] Lustre: DEBUG MARKER: ls -l 1000 files with statahead: 1 sec
[ 8629.751112] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8638.431682] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8665.322449] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files without statahead: 25 sec
[ 8665.741625] Lustre: DEBUG MARKER: ls -l 10000 files without statahead: 25 sec
[ 8680.878113] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files with statahead: 13 sec
[ 8681.296539] Lustre: DEBUG MARKER: ls -l 10000 files with statahead: 13 sec
[ 8681.727080] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l done
[ 8682.053020] Lustre: DEBUG MARKER: ls -l done
[ 8700.514452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ad.sanity: 18 seconds
[ 8700.931535] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ad.sanity: 18 seconds
[ 8701.639316] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8711.572203] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8723.295286] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8737.750158] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8771.094484] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files without statahead: 2 sec
[ 8771.597102] Lustre: DEBUG MARKER: ls -l 100 files without statahead: 2 sec
[ 8772.154460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files with statahead: 0 sec
[ 8772.494359] Lustre: DEBUG MARKER: ls -l 100 files with statahead: 0 sec
[ 8773.195471] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8774.579521] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8778.340301] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files without statahead: 3 sec
[ 8778.677124] Lustre: DEBUG MARKER: ls -l 1000 files without statahead: 3 sec
[ 8780.231381] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files with statahead: 1 sec
[ 8780.571732] Lustre: DEBUG MARKER: ls -l 1000 files with statahead: 1 sec
[ 8781.275761] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8789.643291] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8817.338255] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files without statahead: 26 sec
[ 8817.756084] Lustre: DEBUG MARKER: ls -l 10000 files without statahead: 26 sec
[ 8827.005442] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files with statahead: 8 sec
[ 8827.349479] Lustre: DEBUG MARKER: ls -l 10000 files with statahead: 8 sec
[ 8827.756222] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l done
[ 8828.096792] Lustre: DEBUG MARKER: ls -l done
[ 8846.674030] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ad.sanity: 18 seconds
[ 8847.094850] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ad.sanity: 18 seconds
[ 8847.814541] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8857.261848] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8868.293582] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8877.383017] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8897.871804] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123b: not panic with network error in statahead enqueue \(bug 15027\) ========================================================== 16:10:51 \(1734365451\)
[ 8898.393578] Lustre: DEBUG MARKER: == sanity test 123b: not panic with network error in statahead enqueue (bug 15027) ========================================================== 16:10:51 (1734365451)
[ 8899.692145] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8901.206804] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8901.882498] Autotest: Test running for 145 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8903.354349] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls done
[ 8903.698964] Lustre: DEBUG MARKER: ls done
[ 8907.408139] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123c: Can not initialize inode warning on DNE statahead ========================================================== 16:11:01 \(1734365461\)
[ 8907.754670] Lustre: DEBUG MARKER: == sanity test 123c: Can not initialize inode warning on DNE statahead ========================================================== 16:11:01 (1734365461)
[ 8908.151833] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_123c needs \>= 2 MDTs
[ 8908.492781] Lustre: DEBUG MARKER: SKIP: sanity test_123c needs >= 2 MDTs
[ 8908.923175] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123d: Statahead on striped directories works correctly ========================================================== 16:11:03 \(1734365463\)
[ 8909.282048] Lustre: DEBUG MARKER: == sanity test 123d: Statahead on striped directories works correctly ========================================================== 16:11:03 (1734365463)
[ 8909.570668] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 8909.582956] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 8909.741075] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 8912.119879] Lustre: Unmounted lustre-client
[ 8912.131019] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 8912.143096] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 8912.198906] Lustre: Mounted lustre-client
[ 8915.082776] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123e: statahead with large wide striping == 16:11:09 \(1734365469\)
[ 8915.420509] Lustre: DEBUG MARKER: == sanity test 123e: statahead with large wide striping == 16:11:09 (1734365469)
[ 9027.486492] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123f: Retry mechanism with large wide striping files ========================================================== 16:13:01 \(1734365581\)
[ 9029.063826] Lustre: DEBUG MARKER: == sanity test 123f: Retry mechanism with large wide striping files ========================================================== 16:13:01 (1734365581)
[ 9202.773131] Autotest: Test running for 150 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 9279.347055] LustreError: lustre-OST0004-osc-ffff8961d465b800: operation ost_connect to node 10.240.45.50@tcp failed: rc = -114
[ 9279.349796] LustreError: Skipped 37 previous similar messages
[ 9503.523174] Autotest: Test running for 155 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 9542.491686] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123g: Test for stat-ahead advise ========== 16:21:36 \(1734366096\)
[ 9542.917056] Lustre: DEBUG MARKER: == sanity test 123g: Test for stat-ahead advise ========== 16:21:36 (1734366096)
[ 9543.640528] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9545.155773] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9550.454513] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9552.084952] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123h: Verify statahead work with the fname pattern via du ========================================================== 16:21:46 \(1734366106\)
[ 9552.448432] Lustre: DEBUG MARKER: == sanity test 123h: Verify statahead work with the fname pattern via du ========================================================== 16:21:46 (1734366106)
[ 9628.258285] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9629.286359] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9796.177959] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9797.209959] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9804.602491] Autotest: Test running for 160 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 9816.899342] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123i: Verify statahead work with the fname indexing pattern ========================================================== 16:26:10 \(1734366370\)
[ 9817.232167] Lustre: DEBUG MARKER: == sanity test 123i: Verify statahead work with the fname indexing pattern ========================================================== 16:26:10 (1734366370)
[ 9818.005035] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9819.039824] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9820.510177] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9821.533540] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9822.556940] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9823.579434] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9824.601862] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9825.624087] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9826.647032] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9827.669789] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9828.692058] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9829.714654] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9830.737843] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9831.762212] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9832.785096] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9833.807476] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9834.832036] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9835.854498] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9836.877038] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9837.899085] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9838.921166] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9839.943408] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9840.965954] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9841.988200] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9843.011285] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9844.034424] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9845.057134] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9846.079851] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9847.102603] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9848.124955] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9849.147219] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9850.170370] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9851.193204] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9851.884695] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9852.978822] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9854.481272] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9855.504182] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9856.528473] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9857.550980] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9858.573908] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9859.596961] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9860.620951] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9861.644221] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9862.667738] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9863.691271] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9864.714504] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9865.738694] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9866.763728] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9867.786293] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9868.809971] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9869.833390] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9870.856888] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9871.884604] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9872.912106] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9873.935094] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9874.958160] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9875.980458] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9877.003519] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9878.026305] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9879.050270] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9880.073904] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9881.096577] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9882.119114] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9883.141681] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9884.165085] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9885.187713] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9886.210822] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9888.232085] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9889.255679] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9890.730209] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9892.166708] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9893.357878] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123j: -ENOENT error from batched statahead be handled correctly ========================================================== 16:27:27 \(1734366447\)
[ 9893.698336] Lustre: DEBUG MARKER: == sanity test 123j: -ENOENT error from batched statahead be handled correctly ========================================================== 16:27:27 (1734366447)
[ 9894.097931] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_123j needs \>= 2 MDTs
[ 9894.434203] Lustre: DEBUG MARKER: SKIP: sanity test_123j needs >= 2 MDTs
[ 9894.884708] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123k: Verify statahead work with mdtest shared stat\(\) mode ========================================================== 16:27:28 \(1734366448\)
[ 9895.226142] Lustre: DEBUG MARKER: == sanity test 123k: Verify statahead work with mdtest shared stat() mode ========================================================== 16:27:28 (1734366448)
[ 9904.608668] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9905.632388] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9906.655978] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9907.678303] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9908.700890] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9909.722992] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9910.745008] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9911.768358] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9912.791364] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9913.814099] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9914.836334] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9915.859543] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9916.882087] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9917.904073] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9918.926784] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9919.950069] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9920.974116] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9921.996151] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9923.018836] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9924.042104] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9925.065358] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9926.088032] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9927.110529] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9928.132354] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9929.154551] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9930.177019] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9931.199145] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9932.221171] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9933.243928] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9934.266440] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9935.289137] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9936.311854] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9937.334105] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9939.072448] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123l: Avoid panic when revalidate a local cached entry ========================================================== 16:28:13 \(1734366493\)
[ 9939.418786] Lustre: DEBUG MARKER: == sanity test 123l: Avoid panic when revalidate a local cached entry ========================================================== 16:28:13 (1734366493)
[ 9940.387857] LustreError: 393459:0:(statahead.c:371:sa_put()) cfs_fail_timeout id 1433 sleeping for 35000ms
[ 9975.437142] LustreError: 393459:0:(statahead.c:371:sa_put()) cfs_fail_timeout id 1433 awake
[ 9975.640505] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9976.667918] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9977.690434] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9978.715902] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9979.738350] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9980.763233] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9981.785435] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9982.807472] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9983.829825] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9984.851912] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9985.873979] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9986.895911] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9987.918802] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9988.940846] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9989.963313] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9990.985585] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9992.007692] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9993.029852] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9994.052186] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9995.075111] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9996.097477] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9997.119369] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9998.141386] Lustre: DEBUG MARKER: pgrep ll_sa
[ 9999.163476] Lustre: DEBUG MARKER: pgrep ll_sa
[10000.185899] Lustre: DEBUG MARKER: pgrep ll_sa
[10001.207618] Lustre: DEBUG MARKER: pgrep ll_sa
[10002.229532] Lustre: DEBUG MARKER: pgrep ll_sa
[10003.251631] Lustre: DEBUG MARKER: pgrep ll_sa
[10004.273949] Lustre: DEBUG MARKER: pgrep ll_sa
[10005.297283] Lustre: DEBUG MARKER: pgrep ll_sa
[10006.322246] Lustre: DEBUG MARKER: pgrep ll_sa
[10008.267679] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124a: lru resize ================================================================================================= 16:29:22 \(1734366562\)
[10008.607208] Lustre: DEBUG MARKER: == sanity test 124a: lru resize ================================================================================================= 16:29:22 (1734366562)
[10009.002256] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create 2000 files at \/mnt\/lustre\/d124a.sanity
[10009.342520] Lustre: DEBUG MARKER: create 2000 files at /mnt/lustre/d124a.sanity
[10010.036817] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10012.486594] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10015.695793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark NSDIR=ldlm.namespaces.lustre-MDT0000-mdc-ffff8961d465b800
[10016.028464] Lustre: DEBUG MARKER: NSDIR=ldlm.namespaces.lustre-MDT0000-mdc-ffff8961d465b800
[10016.425166] Lustre: DEBUG MARKER: /usr/sbin/lctl mark NS=ldlm.namespaces.lustre-MDT0000-mdc-ffff8961d465b800
[10016.761814] Lustre: DEBUG MARKER: NS=ldlm.namespaces.lustre-MDT0000-mdc-ffff8961d465b800
[10017.156314] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LRU=2003
[10017.494599] Lustre: DEBUG MARKER: LRU=2003
[10017.889673] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LIMIT=132899
[10018.243119] Lustre: DEBUG MARKER: LIMIT=132899
[10018.611580] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LVF=7961900
[10018.948380] Lustre: DEBUG MARKER: LVF=7961900
[10019.349038] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OLD_LVF=100
[10019.685689] Lustre: DEBUG MARKER: OLD_LVF=100
[10020.087450] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Sleep 50 sec
[10020.437895] Lustre: DEBUG MARKER: Sleep 50 sec
[10070.926564] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Dropped 848 locks in 50s
[10071.352084] Lustre: DEBUG MARKER: Dropped 848 locks in 50s
[10071.760165] Lustre: DEBUG MARKER: /usr/sbin/lctl mark unlink 2000 files at \/mnt\/lustre\/d124a.sanity
[10072.117785] Lustre: DEBUG MARKER: unlink 2000 files at /mnt/lustre/d124a.sanity
[10072.799880] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10074.606885] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10077.129951] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124b: lru resize \(performance test\) ================================================================================= 16:30:31 \(1734366631\)
[10077.472490] Lustre: DEBUG MARKER: == sanity test 124b: lru resize (performance test) ================================================================================= 16:30:31 (1734366631)
[10078.308863] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10082.720100] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10083.892127] Lustre: DEBUG MARKER: /usr/sbin/lctl mark doing ls -la \/mnt\/lustre\/d124b.sanity\/disable_lru_resize 3 times
[10084.234990] Lustre: DEBUG MARKER: doing ls -la /mnt/lustre/d124b.sanity/disable_lru_resize 3 times
[10092.480010] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -la time: 8 seconds
[10092.825833] Lustre: DEBUG MARKER: ls -la time: 8 seconds
[10093.230037] Lustre: DEBUG MARKER: /usr/sbin/lctl mark lru_size = 200
[10093.574682] Lustre: DEBUG MARKER: lru_size = 200
[10094.278799] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10097.920463] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10099.402390] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10103.596102] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10104.783938] Lustre: DEBUG MARKER: /usr/sbin/lctl mark doing ls -la \/mnt\/lustre\/d124b.sanity\/enable_lru_resize 3 times
[10105.126695] Lustre: DEBUG MARKER: doing ls -la /mnt/lustre/d124b.sanity/enable_lru_resize 3 times
[10105.839392] Autotest: Test running for 165 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[10111.653805] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -la time: 5 seconds
[10111.973171] Lustre: DEBUG MARKER: ls -la time: 5 seconds
[10112.371368] Lustre: DEBUG MARKER: /usr/sbin/lctl mark lru_size = 4006
[10112.707913] Lustre: DEBUG MARKER: lru_size = 4006
[10113.095851] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -la is 37% faster with lru resize enabled
[10113.436070] Lustre: DEBUG MARKER: ls -la is 37% faster with lru resize enabled
[10116.164138] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10118.506949] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10119.990676] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124c: LRUR cancel very aged locks ========= 16:31:14 \(1734366674\)
[10120.549378] Lustre: DEBUG MARKER: == sanity test 124c: LRUR cancel very aged locks ========= 16:31:14 (1734366674)
[10143.680201] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124d: cancel very aged locks if lru-resize disabled ========================================================== 16:31:37 \(1734366697\)
[10144.029627] Lustre: DEBUG MARKER: == sanity test 124d: cancel very aged locks if lru-resize disabled ========================================================== 16:31:37 (1734366697)
[10166.693071] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 125: don\'t return EPROTO when a dir has a non-default striping and ACLs ========================================================== 16:32:00 \(1734366720\)
[10167.040828] Lustre: DEBUG MARKER: == sanity test 125: don't return EPROTO when a dir has a non-default striping and ACLs ========================================================== 16:32:00 (1734366720)
[10168.833498] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 126: check that the fsgid provided by the client is taken into account ========================================================== 16:32:02 \(1734366722\)
[10169.182647] Lustre: DEBUG MARKER: == sanity test 126: check that the fsgid provided by the client is taken into account ========================================================== 16:32:02 (1734366722)
[10170.965655] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 127a: verify the client stats are sane ==== 16:32:05 \(1734366725\)
[10171.308012] Lustre: DEBUG MARKER: == sanity test 127a: verify the client stats are sane ==== 16:32:05 (1734366725)
[10173.361417] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 127b: verify the llite client stats are sane ========================================================== 16:32:07 \(1734366727\)
[10173.692909] Lustre: DEBUG MARKER: == sanity test 127b: verify the llite client stats are sane ========================================================== 16:32:07 (1734366727)
[10175.506197] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 127c: test llite extent stats with regular & mmap i\/o ========================================================== 16:32:09 \(1734366729\)
[10175.850823] Lustre: DEBUG MARKER: == sanity test 127c: test llite extent stats with regular
[10199.179732] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 128: interactive lfs for 2 consecutive find\'s ========================================================== 16:32:33 \(1734366753\)
[10199.520457] Lustre: DEBUG MARKER: == sanity test 128: interactive lfs for 2 consecutive find's ========================================================== 16:32:33 (1734366753)
[10201.305554] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 129: test directory size limit ================================================================================== 16:32:35 \(1734366755\)
[10201.649624] Lustre: DEBUG MARKER: == sanity test 129: test directory size limit ================================================================================== 16:32:35 (1734366755)
[10210.969268] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10211.959477] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10214.692993] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130a: FIEMAP \(1-stripe file\) ============== 16:32:48 \(1734366768\)
[10215.011416] Lustre: DEBUG MARKER: == sanity test 130a: FIEMAP (1-stripe file) ============== 16:32:48 (1734366768)
[10216.855181] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130b: FIEMAP \(7-stripe file\) ============== 16:32:50 \(1734366770\)
[10217.200500] Lustre: DEBUG MARKER: == sanity test 130b: FIEMAP (7-stripe file) ============== 16:32:50 (1734366770)
[10227.969641] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130c: FIEMAP \(2-stripe file with hole\) ==== 16:33:02 \(1734366782\)
[10228.303515] Lustre: DEBUG MARKER: == sanity test 130c: FIEMAP (2-stripe file with hole) ==== 16:33:02 (1734366782)
[10230.803437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130d: FIEMAP \(N-stripe file\) ============== 16:33:04 \(1734366784\)
[10231.138972] Lustre: DEBUG MARKER: == sanity test 130d: FIEMAP (N-stripe file) ============== 16:33:04 (1734366784)
[10234.455699] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130e: FIEMAP \(test continuation FIEMAP calls\) ========================================================== 16:33:08 \(1734366788\)
[10234.797225] Lustre: DEBUG MARKER: == sanity test 130e: FIEMAP (test continuation FIEMAP calls) ========================================================== 16:33:08 (1734366788)
[10245.334901] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130f: FIEMAP \(unstriped file\) ============= 16:33:19 \(1734366799\)
[10245.673515] Lustre: DEBUG MARKER: == sanity test 130f: FIEMAP (unstriped file) ============= 16:33:19 (1734366799)
[10247.414762] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130g: FIEMAP \(overstripe file\) ============ 16:33:21 \(1734366801\)
[10247.747964] Lustre: DEBUG MARKER: == sanity test 130g: FIEMAP (overstripe file) ============ 16:33:21 (1734366801)
[10379.168834] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130h: FIEMAP deadlock ===================== 16:35:33 \(1734366933\)
[10379.516655] Lustre: DEBUG MARKER: == sanity test 130h: FIEMAP deadlock ===================== 16:35:33 (1734366933)
[10392.377094] LustreError: 410926:0:(osc_object.c:274:osc_object_fiemap()) cfs_fail_timeout id 418 sleeping for 5000ms
[10397.478294] LustreError: 410926:0:(osc_object.c:274:osc_object_fiemap()) cfs_fail_timeout id 418 awake
[10399.268609] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130i: FIEMAP \(DoM file\) =================== 16:35:53 \(1734366953\)
[10399.614848] Lustre: DEBUG MARKER: == sanity test 130i: FIEMAP (DoM file) =================== 16:35:53 (1734366953)
[10407.328256] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131a: test iov\'s crossing stripe boundary for writev\/readv ========================================================== 16:36:01 \(1734366961\)
[10407.671658] Lustre: DEBUG MARKER: == sanity test 131a: test iov's crossing stripe boundary for writev/readv ========================================================== 16:36:01 (1734366961)
[10409.540465] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131b: test append writev ================== 16:36:03 \(1734366963\)
[10409.896146] Lustre: DEBUG MARKER: == sanity test 131b: test append writev ================== 16:36:03 (1734366963)
[10411.858550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131c: test read\/write on file w\/o objects ========================================================== 16:36:05 \(1734366965\)
[10412.201339] Lustre: DEBUG MARKER: == sanity test 131c: test read/write on file w/o objects ========================================================== 16:36:05 (1734366965)
[10413.974977] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131d: test short read ===================== 16:36:08 \(1734366968\)
[10414.319924] Lustre: DEBUG MARKER: == sanity test 131d: test short read ===================== 16:36:08 (1734366968)
[10416.160417] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131e: test read hitting hole ============== 16:36:10 \(1734366970\)
[10416.507282] Lustre: DEBUG MARKER: == sanity test 131e: test read hitting hole ============== 16:36:10 (1734366970)
[10417.562948] Autotest: Test running for 170 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[10418.392782] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133a: Verifying MDT stats ================================================================================================== 16:36:12 \(1734366972\)
[10418.740070] Lustre: DEBUG MARKER: == sanity test 133a: Verifying MDT stats ================================================================================================== 16:36:12 (1734366972)
[10426.236663] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133b: Verifying extra MDT stats ============================================================================================ 16:36:20 \(1734366980\)
[10426.579323] Lustre: DEBUG MARKER: == sanity test 133b: Verifying extra MDT stats ============================================================================================ 16:36:20 (1734366980)
[10437.894989] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133c: Verifying OST stats ================================================================================================== 16:36:31 \(1734366991\)
[10438.235257] Lustre: DEBUG MARKER: == sanity test 133c: Verifying OST stats ================================================================================================== 16:36:31 (1734366991)
[10460.731612] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133d: Verifying rename_stats ================================================================================================== 16:36:54 \(1734367014\)
[10461.080129] Lustre: DEBUG MARKER: == sanity test 133d: Verifying rename_stats ================================================================================================== 16:36:54 (1734367014)
[10462.598828] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10463.730967] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10470.112754] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133e: Verifying OST {read,write}_bytes nid stats =========================================================================== 16:37:04 \(1734367024\)
[10470.429631] Lustre: DEBUG MARKER: == sanity test 133e: Verifying OST read_bytes write_bytes nid stats =========================================================================== 16:37:04 (1734367024)
[10475.852746] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133f: Check reads\/writes of client lustre proc files with bad area io ========================================================== 16:37:09 \(1734367029\)
[10476.185054] Lustre: DEBUG MARKER: == sanity test 133f: Check reads/writes of client lustre proc files with bad area io ========================================================== 16:37:09 (1734367029)
[10476.328231] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[10479.027693] Lustre: 422015:0:(libcfs_string.c:161:cfs_str2mask()) unknown mask ''.
[10479.027693] mask usage: [+|-]<all|type> ...
[10479.034187] ------------[ cut here ]------------
[10479.035082] WARNING: CPU: 0 PID: 422015 at mm/page_alloc.c:5396 __alloc_pages+0x2b0/0x330
[10479.036501] Modules linked in: tls tcp_diag inet_diag lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 nfsv4 nfs fscache netfs intel_rapl_msr intel_rapl_common joydev input_leds mac_hid serio_raw qemu_fw_cfg binfmt_misc sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua ipmi_devintf nfsd ipmi_msghandler msr auth_rpcgss nfs_acl efi_pstore lockd drm grace sunrpc ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd virtio_net net_failover cryptd floppy failover i2c_piix4 psmouse virtio_blk pata_acpi [last unloaded: obd_test]
[10479.048022] CPU: 0 PID: 422015 Comm: badarea_io Kdump: loaded Tainted: G OE 5.15.0-94-generic #104-Ubuntu
[10479.049751] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[10479.050730] RIP: 0010:__alloc_pages+0x2b0/0x330
[10479.051532] Code: 48 8b 04 25 c0 fb 01 00 48 05 a0 0d 00 00 41 be 01 00 00 00 48 89 45 b0 e9 45 fe ff ff 41 81 e4 00 20 00 00 0f 85 60 ff ff ff <0f> 0b e9 59 ff ff ff 31 c0 e9 19 fe ff ff f7 c2 00 00 08 00 75 53
[10479.054453] RSP: 0018:ffffaebc4515bbe0 EFLAGS: 00010246
[10479.055354] RAX: 0000000000000000 RBX: ffffffffa0f3cda0 RCX: 0000000000000000
[10479.056545] RDX: 0000000000000000 RSI: 0000000000000013 RDI: 0000000000040cc0
[10479.057729] RBP: ffffaebc4515bc38 R08: ffffaebc4515be40 R09: 0000000000000000
[10479.058902] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[10479.060079] R13: 0000000000000013 R14: 0000000000000000 R15: 0000000000000000
[10479.061315] FS: 00007fb8f95e4740(0000) GS:ffff89623fc00000(0000) knlGS:0000000000000000
[10479.062672] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10479.063667] CR2: 00007ffe2efac000 CR3: 00000000a559a005 CR4: 00000000003706f0
[10479.064867] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10479.066073] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[10479.067280] Call Trace:
[10479.067773] <TASK>
[10479.068229] ? show_trace_log_lvl+0x1d6/0x2ea
[10479.069034] ? show_trace_log_lvl+0x1d6/0x2ea
[10479.069808] ? alloc_pages+0x9e/0x1e0
[10479.070499] ? show_regs.part.0+0x23/0x29
[10479.071220] ? show_regs.cold+0x8/0xd
[10479.071887] ? __alloc_pages+0x2b0/0x330
[10479.072591] ? __warn+0x8c/0x100
[10479.073204] ? __alloc_pages+0x2b0/0x330
[10479.073915] ? report_bug+0xa4/0xd0
[10479.074589] ? handle_bug+0x39/0x90
[10479.075252] ? exc_invalid_op+0x19/0x70
[10479.075943] ? asm_exc_invalid_op+0x1b/0x20
[10479.076690] ? __alloc_pages+0x2b0/0x330
[10479.077395] alloc_pages+0x9e/0x1e0
[10479.078036] kmalloc_order+0x2f/0xd0
[10479.078710] kmalloc_order_trace+0x1d/0x90
[10479.079442] __kmalloc_track_caller+0x2c1/0x340
[10479.080233] ? proc_dobitmasks+0xa9/0x110 [libcfs]
[10479.081107] memdup_user_nul+0x26/0xa0
[10479.081790] proc_dobitmasks+0xa9/0x110 [libcfs]
[10479.082608] lnet_debugfs_write+0x3e/0x90 [libcfs]
[10479.083446] full_proxy_write+0x5c/0x90
[10479.084152] vfs_write+0xc7/0x270
[10479.084777] ksys_write+0x67/0xf0
[10479.085376] __x64_sys_write+0x19/0x20
[10479.086051] do_syscall_64+0x5c/0xc0
[10479.086706] ? do_syscall_64+0x69/0xc0
[10479.087378] ? ksys_write+0x67/0xf0
[10479.088016] ? exit_to_user_mode_prepare+0x37/0xb0
[10479.088863] ? syscall_exit_to_user_mode+0x35/0x50
[10479.089689] ? __x64_sys_write+0x19/0x20
[10479.090383] ? do_syscall_64+0x69/0xc0
[10479.091062] ? syscall_exit_to_user_mode+0x35/0x50
[10479.091891] ? do_syscall_64+0x69/0xc0
[10479.092563] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[10479.093420] RIP: 0033:0x7fb8f96fb887
[10479.094084] Code: 10 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[10479.096971] RSP: 002b:00007ffe2efa7fd8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[10479.098217] RAX: ffffffffffffffda RBX: 00007ffe2efa8118 RCX: 00007fb8f96fb887
[10479.099391] RDX: 0000000060000000 RSI: 00007ffe2efa7fe4 RDI: 0000000000000003
[10479.100565] RBP: 0000000000000002 R08: 00007fb8f9802f10 R09: 00007fb8f9820040
[10479.101734] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffe2efa8118
[10479.102914] R13: 000055743a7ad0e0 R14: 000055743a7afda0 R15: 00007fb8f9854040
[10479.104080] </TASK>
[10479.104528] ---[ end trace 0e2302b25e262817 ]---
[10479.108204] Lustre: DEBUG MARKER:
[10479.108924] Lustre: DEBUG MARKER:
[10496.642522] Lustre: 4442:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734367034/real 1734367034] req@ffff8961d46ee3c0 x1818605516989440/t0(0) o400->MGC10.240.38.186@tcp@10.240.38.186@tcp:26/25 lens 224/224 e 0 to 1 dl 1734367050 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[10496.642558] Lustre: lustre-MDT0000-mdc-ffff8961d465b800: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10496.653750] Lustre: 4442:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[10496.658176] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[10496.662674] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f4940a70 to 0x5d837553f4d063bd
[10496.665038] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[10500.738573] Lustre: 4442:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734367039/real 1734367039] req@ffff8961d43f9a00 x1818605516990720/t0(0) o400->lustre-MDT0000-mdc-ffff8961d465b800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734367055 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[10505.858550] Lustre: 4443:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734367044/real 1734367044] req@ffff8961d46ef740 x1818605516991872/t0(0) o400->lustre-MDT0000-mdc-ffff8961d465b800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734367060 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[10506.089246] Lustre: lustre-MDT0000-mdc-ffff8961d465b800: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[10506.320333] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[10507.491298] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[10507.901183] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[10508.183148] Lustre: DEBUG MARKER: lctl get_param -n at_max
[10508.436830] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[10508.875098] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[10510.921890] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[10510.921890] if [ $running -ne 0 ] ; then
[10510.921890] echo Stopping client $(hostname) /mnt/lustre opts:;
[10510.921890] lsof /mnt/lustre || need_kill=no;
[10510.921890] if [ x != x -a x$need_kill != xno ]; then
[10510.921890] pids=$(lsof -t /mnt/lustre | sort -u);
[10510.921890] if
[10510.978558] Lustre: 4442:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734367049/real 1734367049] req@ffff8961d43fb0c0 x1818605516993024/t0(0) o400->lustre-MDT0000-mdc-ffff8961d465b800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734367065 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[10517.931375] Lustre: Unmounted lustre-client
[10518.121601] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[10518.121601] if [ $running -ne 0 ] ; then
[10518.121601] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[10518.121601] lsof /mnt/lustre2 || need_kill=no;
[10518.121601] if [ x != x -a x$need_kill != xno ]; then
[10518.121601] pids=$(lsof -t /mnt/lustre2 | sort -u);
[10518.121601]
[10581.535684] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[10581.535684] lctl dl | grep ' ST ' || true
[10581.948398] Key type lgssc unregistered
[10582.022997] LNet: 424094:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[10582.042940] LNet: Removed LNI 10.240.44.153@tcp
[10582.267263] Key type .llcrypt unregistered
[10582.268141] Key type ._llcrypt unregistered
[10582.713348] Key type ._llcrypt registered
[10582.714221] Key type .llcrypt registered
[10582.803078] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[10583.718258] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[10583.785648] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[10583.786819] LNet: Accept all, port 7988
[10585.326771] Key type lgssc registered
[10585.530228] Lustre: Echo OBD driver; http://www.lustre.org/
[10589.569320] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[10589.569320] mpts=$(mount | grep -c /mnt/lustre' ');
[10589.569320] if [ $running -ne $mpts ]; then
[10589.569320] echo $(hostname) env are INSANE!;
[10589.569320] exit 1;
[10589.569320] fi
[10590.314201] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[10590.314201] mpts=$(mount | grep -c /mnt/lustre2' ');
[10590.314201] if [ $running -ne $mpts ]; then
[10590.314201] echo $(hostname) env are INSANE!;
[10590.314201] exit 1;
[10590.314201] fi
[10697.608485] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[10697.620595] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[10703.761482] Lustre: Mounted lustre-client
[10703.964709] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[10704.704829] Lustre: DEBUG MARKER:
[10704.704829] running=$(mount | grep -c /mnt/lustre' ');
[10704.704829] rc=0;
[10704.704829] if [ $running -eq 0 ] ; then
[10704.704829] mkdir -p /mnt/lustre;
[10704.704829] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[10704.704829] rc=$?;
[10704.704829] fi;
[10704.704829] exit $rc
[10705.421068] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[10710.041408] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[10710.386557] Lustre: DEBUG MARKER: Using TIMEOUT=20
[10710.800275] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[10711.618432] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[10712.052107] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[10712.064365] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[10718.763188] Autotest: Test running for 175 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[10722.613033] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133g: Check reads\/writes of server lustre proc files with bad area io ========================================================== 16:41:16 \(1734367276\)
[10722.961221] Lustre: DEBUG MARKER: == sanity test 133g: Check reads/writes of server lustre proc files with bad area io ========================================================== 16:41:16 (1734367276)
[10729.347255] Lustre: lustre-OST0000-osc-ffff8961851de800: disconnect after 20s idle
[10738.135935] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[10738.135935] if [ $running -ne 0 ] ; then
[10738.135935] echo Stopping client $(hostname) /mnt/lustre opts:;
[10738.135935] lsof /mnt/lustre || need_kill=no;
[10738.135935] if [ x != x -a x$need_kill != xno ]; then
[10738.135935] pids=$(lsof -t /mnt/lustre | sort -u);
[10738.135935] if
[10739.029582] Lustre: Unmounted lustre-client
[10739.605614] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[10739.605614] if [ $running -ne 0 ] ; then
[10739.605614] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[10739.605614] lsof /mnt/lustre2 || need_kill=no;
[10739.605614] if [ x != x -a x$need_kill != xno ]; then
[10739.605614] pids=$(lsof -t /mnt/lustre2 | sort -u);
[10739.605614]
[10776.105553] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1 || true
[10777.581887] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[10777.581887] lctl dl | grep ' ST ' || true
[10777.957104] Key type lgssc unregistered
[10778.023453] LNet: 429681:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[10779.059496] LNet: Removed LNI 10.240.44.153@tcp
[10779.299637] Key type .llcrypt unregistered
[10779.300515] Key type ._llcrypt unregistered
[10779.704390] Key type ._llcrypt registered
[10779.705245] Key type .llcrypt registered
[10779.793086] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[10780.636087] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[10780.702212] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[10780.703399] LNet: Accept all, port 7988
[10782.243294] Key type lgssc registered
[10782.432175] Lustre: Echo OBD driver; http://www.lustre.org/
[10786.421688] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[10786.421688] mpts=$(mount | grep -c /mnt/lustre' ');
[10786.421688] if [ $running -ne $mpts ]; then
[10786.421688] echo $(hostname) env are INSANE!;
[10786.421688] exit 1;
[10786.421688] fi
[10787.081105] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[10787.081105] mpts=$(mount | grep -c /mnt/lustre2' ');
[10787.081105] if [ $running -ne $mpts ]; then
[10787.081105] echo $(hostname) env are INSANE!;
[10787.081105] exit 1;
[10787.081105] fi
[10902.243422] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[10902.261273] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[10903.314404] Lustre: Mounted lustre-client
[10903.519937] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[10904.168518] Lustre: DEBUG MARKER:
[10904.168518] running=$(mount | grep -c /mnt/lustre' ');
[10904.168518] rc=0;
[10904.168518] if [ $running -eq 0 ] ; then
[10904.168518] mkdir -p /mnt/lustre;
[10904.168518] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[10904.168518] rc=$?;
[10904.168518] fi;
[10904.168518] exit $rc
[10904.907777] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[10909.490835] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[10909.836412] Lustre: DEBUG MARKER: Using TIMEOUT=20
[10910.229814] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[10911.070885] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[10918.102038] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133h: Proc files should end with newlines ========================================================== 16:44:32 \(1734367472\)
[10918.444295] Lustre: DEBUG MARKER: == sanity test 133h: Proc files should end with newlines ========================================================== 16:44:32 (1734367472)
[10918.670567] Lustre: DEBUG MARKER:
[10918.702762] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_early_margin | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.720934] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_extra | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.738376] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.755895] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.773427] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.791232] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.808810] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_peer_on_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.826253] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_raw_pointers | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.843898] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param dump_on_eviction | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.861232] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param dump_on_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.878654] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param enable_stats_header | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.896252] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param evict_multiplier | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.913631] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param jobid_name | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.931019] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param jobid_this_session | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.948514] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param jobid_var | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.965957] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lbug_on_eviction | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10918.983429] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.cancel_unused_locks_before_replay | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.001265] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.019203] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.037258] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.055331] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.073303] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.091251] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.109089] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.126977] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.144960] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.162967] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.180998] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.38.186@tcp.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.199028] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.217295] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.235801] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.253921] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.272198] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.290446] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.308764] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.326972] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.345261] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.363454] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.381977] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.400386] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.418671] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.437018] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.455289] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.473522] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.491780] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.509863] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.528182] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.546352] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.564698] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.582981] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.606322] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.624507] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.642678] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.660948] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.679358] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.697424] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.715570] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.733678] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.751699] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.770081] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.788403] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.806549] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.824977] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.843264] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.861492] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.879734] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.897860] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.916025] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.934304] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.952562] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.970974] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10919.989268] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.007407] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.025565] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.043946] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.062237] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.080414] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.098555] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.116682] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.134805] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.152978] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.171354] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.189576] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.207769] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.225985] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.244395] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.262589] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.280953] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.299124] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.317269] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.335371] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.353482] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.371906] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.390287] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.408507] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.426707] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.444830] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.462906] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.481106] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.499204] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.517272] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.535435] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.553581] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.571930] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.590242] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.608266] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.627047] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.645333] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.663417] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.681575] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.699686] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.717641] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.735627] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.753823] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.772001] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.790146] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff896191de2800.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.808098] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.high_priority_ratio | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.825752] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.threads_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.843359] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.threads_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.860973] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_delay_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.878770] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_delay_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.896382] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_delay_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.913903] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_policies | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.931453] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.req_buffer_history_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.949218] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.req_buffers_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.966904] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10920.984427] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.dump_granted_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.001857] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.dump_namespaces | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.019098] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.checksum_pages | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.036821] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.054384] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.default_easize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.072107] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.enable_filename_encryption | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.089870] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.enable_setstripe_gid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.107565] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.enable_statahead_fname | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.125410] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.fast_read | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.143101] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.file_heat | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.160857] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.filename_enc_use_old_base64 | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.178727] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.foreign_symlink_enable | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.196637] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.foreign_symlink_prefix | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.214375] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.foreign_symlink_upcall | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.232172] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.foreign_symlink_upcall_info | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.249996] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.heat_decay_percentage | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.267828] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.heat_period_second | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.285501] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.hybrid_io | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.303210] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.hybrid_io_read_threshold_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.321443] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.hybrid_io_write_threshold_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.339575] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.inode_cache | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.357411] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.intent_mkdir | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.375135] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lazystatfs | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.392879] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.qos_maxage | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.410858] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.qos_prio_free | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.428803] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.qos_threshold_rr | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.446817] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.465297] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.483778] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.502249] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.520737] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.539385] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.557819] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.576254] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.594735] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.613457] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.max_mod_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.632055] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.650544] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lmv.target_obds.lustre-MDT0000-mdc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.669334] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.stripecount | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.687175] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.stripeoffset | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.705128] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.stripesize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.723092] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.stripetype | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.740967] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.759427] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.777959] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.796716] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.815329] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.834224] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.852771] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.872429] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.891616] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.910830] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.929828] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.948821] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.967807] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10921.986611] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.005579] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.024951] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.043894] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0000-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.062855] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.081709] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.100567] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.119420] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.138099] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.157058] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.175958] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.194963] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.213875] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.233060] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.252151] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.271227] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.290219] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.309192] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.328162] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.347282] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.365958] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0001-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.384851] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.403943] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.423416] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.442559] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.461663] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.481051] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.500058] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.519204] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.538031] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.557147] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.576362] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.595367] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.614589] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.633661] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.652778] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.672033] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.691044] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0002-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.709973] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.728741] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.747455] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.766295] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.785007] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.803893] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.822738] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.841728] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.860546] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.879528] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.898421] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.917211] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.936100] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.954886] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.973806] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10922.992896] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.011795] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0003-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.030551] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.049304] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.068247] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.086899] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.105600] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.124373] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.143092] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.162066] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.180938] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.199953] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.218694] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.237527] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.256464] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.275367] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.294320] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.313557] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.332777] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0004-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.351898] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.370815] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.389723] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.408610] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.427497] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.446480] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.465493] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.484569] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.503399] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.522472] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.541321] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.560193] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.579057] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.597908] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.616696] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.635836] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.654719] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0005-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.673627] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.692423] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.711334] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.730207] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.748968] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.767905] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.786544] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.805462] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.824529] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.843931] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.862688] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.881551] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.900542] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.919665] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.938924] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.957939] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.976837] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.lov.target_obds.lustre-OST0006-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10923.995874] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.max_read_ahead_async_active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.014258] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.max_read_ahead_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.032611] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.max_read_ahead_per_file_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.051242] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.max_read_ahead_whole_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.069784] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.opencache_max_ms | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.088099] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.opencache_threshold_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.106618] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.opencache_threshold_ms | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.124980] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.parallel_dio | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.143258] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.pcc_async_affinity | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.161794] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.pcc_async_threshold | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.180452] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.pcc_mode | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.198829] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.read_ahead_async_file_threshold_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.217452] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.read_ahead_range_kb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.235888] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.stat_blocksize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.254164] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_agl | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.272481] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_batch_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.290732] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_fname_match_hit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.309123] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_fname_predict_hit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.327559] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.345904] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.364251] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_running_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.382621] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.401218] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statfs_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.419909] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statfs_project | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.438460] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.stats_track_gid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.456917] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.stats_track_pid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.475001] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.stats_track_ppid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.493262] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.tiny_write | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.511608] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.unaligned_dio | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.530104] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.xattr_cache | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.548630] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.enable_mlock_pages | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.567760] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.extents_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.586818] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.extents_stats_per_process | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.605454] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.max_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.623989] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.nosquash_nids | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.642231] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.offset_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.660504] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.pcc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.678573] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.read_ahead_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.696731] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.root_squash | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.714946] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.statahead_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.733109] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.751153] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.769417] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff896191de2800.unstable_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.788310] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.qos_maxage | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.806712] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.qos_prio_free | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.824977] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.qos_threshold_rr | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.843231] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.861934] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.880716] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.899496] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.918201] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.937072] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.955948] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.975003] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10924.993900] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.012826] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.max_mod_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.031827] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.050823] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.target_obds.lustre-MDT0000-mdc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.069945] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.md_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.088102] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff896191de2800.qos_exclude_prefixes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.106374] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.stripecount | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.124481] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.stripeoffset | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.142989] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.stripesize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.161117] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.stripetype | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.179222] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.198096] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.217086] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.235867] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.254851] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.273809] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.292518] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.311455] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.330263] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.349243] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.368186] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.387227] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.406185] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.425034] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.443955] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.463058] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.481968] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0000-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.501033] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.519778] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.538790] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.557623] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.576419] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.595592] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.614564] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.633806] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.652917] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.671901] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.690883] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.709732] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.728583] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.747390] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.766386] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.785867] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.804958] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0001-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.823946] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.842777] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.861587] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.880497] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.899224] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.918173] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.937162] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.956245] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.977211] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10925.996404] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.015282] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.034113] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.053116] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.072141] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.091031] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.110128] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.129116] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0002-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.148327] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.167161] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.186314] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.205507] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.224671] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.243613] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.262575] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.281791] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.300939] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.320077] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.338937] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.357943] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.376969] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.400594] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.419533] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.441568] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.460482] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0003-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.479465] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.498463] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.517452] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.536242] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.554869] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.573888] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.592801] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.611811] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.630631] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.649642] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.668572] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.687451] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.706355] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.725198] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.744169] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.763271] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.782285] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0004-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.801174] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.819982] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.838773] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.857475] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.876137] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.894919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.913703] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.932641] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.951320] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.970145] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10926.988912] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.007603] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.026547] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.045248] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.064164] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.083068] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.101928] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0005-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.120709] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.139415] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.158403] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.177255] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.196348] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.215322] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.234278] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.253311] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.272175] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.291145] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.309946] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.328819] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.347834] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.366752] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.385901] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.405153] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.424190] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff896191de2800.target_obds.lustre-OST0006-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.443128] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.460770] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.478918] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.497283] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.515446] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.533597] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.552124] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.570487] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.589033] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.607341] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.625823] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.max_mod_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.644357] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.662648] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.681268] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.699498] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.717681] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.batch_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.736008] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.754496] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.772719] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.790961] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.809225] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.md_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.827389] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.mdc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.845519] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.mdc_dom_min_repsize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.863796] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.mdc_lsom | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.882066] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.mdc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.900203] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.918432] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.936665] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.38.186@tcp.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.955079] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.38.186@tcp.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.973097] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.38.186@tcp.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10927.991034] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.38.186@tcp.dynamic_nids | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.009055] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.38.186@tcp.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.027409] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.38.186@tcp.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.045382] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.063852] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.081921] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.100203] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.118470] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.136632] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.155213] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.173502] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.191684] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.210079] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.228162] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.246369] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.264586] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.282815] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.301059] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.319498] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.337752] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.355919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.374129] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.392207] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.410640] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.429049] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.447473] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.465999] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.484465] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.503207] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.521660] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.539882] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.558183] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.576297] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.594527] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.612679] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.630839] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.649373] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.667423] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.685795] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.704006] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.722335] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.740519] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.759072] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.777314] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.795525] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.813794] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.832421] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.850902] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.869276] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.887452] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.905628] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.923835] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.941964] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.960219] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.978606] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10928.996755] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.014754] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.031788] Lustre: lustre-OST0000-osc-ffff896191de2800: disconnect after 21s idle
[10929.035292] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.053584] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.071895] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.090033] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.108196] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.126415] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.144554] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.162816] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.181130] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.199744] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.217929] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.236289] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.254499] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.272779] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.291220] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.309462] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.327803] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.347412] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.365660] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.383945] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.402077] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.420246] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.438472] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.456824] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.475020] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.493552] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.511873] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.530138] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.548574] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.566924] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.585372] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.603611] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.621805] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.639980] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.658120] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.676552] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.695218] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.713720] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.732137] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.750516] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.768718] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.787060] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.805530] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.823740] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.842063] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.861707] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.880110] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.898603] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.916845] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.935241] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.954192] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.972580] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10929.990726] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.009098] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.027442] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.045799] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.064369] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.082994] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.101305] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.119937] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.138785] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.157919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.176953] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.195876] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.214161] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.232672] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.251241] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.269860] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.288345] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.306892] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.325400] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.343963] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.362508] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.382151] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.400627] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.419059] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.437940] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.456749] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.475408] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.494235] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.512634] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.531055] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.549186] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.567243] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.585334] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.603423] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.621547] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.639408] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.657476] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.675427] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.693339] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.711383] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.729461] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.747741] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.765970] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.784114] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.802211] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.820293] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.838360] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.856349] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.874644] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.893952] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.912009] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.930107] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.948097] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.966009] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10930.984098] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.002199] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.020133] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.038261] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.043948] Lustre: lustre-OST0001-osc-ffff896191de2800: disconnect after 23s idle
[10931.045270] Lustre: Skipped 5 previous similar messages
[10931.058679] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.076816] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.095344] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.113727] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.131902] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.150065] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.168275] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.186626] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.204969] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.223484] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.241720] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.260293] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.278720] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.296832] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.314759] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.332705] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.350783] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.369039] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.387741] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.407420] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.425739] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.443970] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.462049] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.480435] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.499520] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.518112] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.536563] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.555154] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.573513] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.591744] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.610118] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.628322] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff896191de2800.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.646796] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ping_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.664631] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.682082] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_backoff | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.699681] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_max_delay_centisecs | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.717441] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_min_delay_centisecs | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.735271] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_ratelimit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.752918] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param daemon_file | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.770645] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.788457] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_log_upcall | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.805852] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_marker | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.823320] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.840807] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_path | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.858527] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param dump_kernel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.876211] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fail_err | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.893829] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fail_loc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.911733] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fail_val | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.929429] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param force_lbug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.946919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param nis | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.964629] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param panic_on_lbug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.982158] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param peers | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10931.999758] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param portal_rotor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.017553] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param printk | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.035013] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.052668] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param subsystem_debug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.070610] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param watchdog_ratelimit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.088497] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fld.cli-lustre-clilmv-ffff896191de2800.cache_flush | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.106806] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fld.cli-lustre-clilmv-ffff896191de2800.hash | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.125053] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-MDT0000-mdc-ffff896191de2800.space | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.142962] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-MDT0000-mdc-ffff896191de2800.width | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.160959] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilmv-ffff896191de2800.space | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.178865] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilmv-ffff896191de2800.width | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.197126] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilov-ffff896191de2800.space | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.215545] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilov-ffff896191de2800.width | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.233852] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.init_channel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.251891] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.lgss_keyring.debug_level | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.270105] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.gss_check_upcall_ns | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.288041] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.krb5_allow_old_client_csum | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.306071] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsc_info | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.323708] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_acquire_expire | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.341342] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_entry_expire | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.359202] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_info | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10932.377152] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_upcall | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) print FILENAME}'
[10951.555921] Lustre: lustre-OST0002-osc-ffff896191de2800: disconnect after 22s idle
[11020.096716] Autotest: Test running for 180 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[11278.188763] Autotest: Test running for 185 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[11470.558611] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 134a: Server reclaims locks when reaching lock_reclaim_threshold ========================================================== 16:53:44 \(1734368024\)
[11470.909106] Lustre: DEBUG MARKER: == sanity test 134a: Server reclaims locks when reaching lock_reclaim_threshold ========================================================== 16:53:44 (1734368024)
[11471.673107] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11473.163174] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11486.548474] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11487.893084] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11490.472888] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 134b: Server rejects lock request when reaching lock_limit_mb ========================================================== 16:54:04 \(1734368044\)
[11490.830082] Lustre: DEBUG MARKER: == sanity test 134b: Server rejects lock request when reaching lock_limit_mb ========================================================== 16:54:04 (1734368044)
[11493.088482] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11513.739706] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11514.757420] Lustre: lustre-OST0002-osc-ffff896191de2800: disconnect after 24s idle
[11514.758866] Lustre: Skipped 4 previous similar messages
[11515.390947] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11516.288883] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11518.795896] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_135 skipping SLOW test 135
[11519.135752] Lustre: DEBUG MARKER: SKIP: sanity test_135 skipping SLOW test 135
[11519.540414] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_136 skipping SLOW test 136
[11519.874582] Lustre: DEBUG MARKER: SKIP: sanity test_136 skipping SLOW test 136
[11520.291785] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 140: Check reasonable stack depth \(shouldn\'t LBUG\) ============================================================== 16:54:34 \(1734368074\)
[11520.639075] Lustre: DEBUG MARKER: == sanity test 140: Check reasonable stack depth (shouldn't LBUG) ============================================================== 16:54:34 (1734368074)
[11525.556270] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150a: truncate\/append tests =============== 16:54:39 \(1734368079\)
[11525.903010] Lustre: DEBUG MARKER: == sanity test 150a: truncate/append tests =============== 16:54:39 (1734368079)
[11526.625901] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[11526.638125] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[11526.794580] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[11550.597537] Lustre: lustre-OST0003-osc-ffff896191de2800: disconnect after 24s idle
[11552.568150] Lustre: Unmounted lustre-client
[11552.577919] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[11552.589252] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[11552.649570] Lustre: Mounted lustre-client
[11563.236075] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150b: Verify fallocate \(prealloc\) functionality ========================================================== 16:55:17 \(1734368117\)
[11563.581177] Lustre: DEBUG MARKER: == sanity test 150b: Verify fallocate (prealloc) functionality ========================================================== 16:55:17 (1734368117)
[11578.886818] Autotest: Test running for 190 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[11585.605972] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150bb: Verify fallocate modes both zero space ========================================================== 16:55:39 \(1734368139\)
[11585.950444] Lustre: DEBUG MARKER: == sanity test 150bb: Verify fallocate modes both zero space ========================================================== 16:55:39 (1734368139)
[11608.324739] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150c: Verify fallocate Size and Blocks ==== 16:56:02 \(1734368162\)
[11608.748060] Lustre: DEBUG MARKER: == sanity test 150c: Verify fallocate Size and Blocks ==== 16:56:02 (1734368162)
[11624.246500] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150d: Verify fallocate Size and Blocks - Non zero start ========================================================== 16:56:18 \(1734368178\)
[11624.679618] Lustre: DEBUG MARKER: == sanity test 150d: Verify fallocate Size and Blocks - Non zero start ========================================================== 16:56:18 (1734368178)
[11639.836776] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150e: Verify 60% of available OST space consumed by fallocate ========================================================== 16:56:33 \(1734368193\)
[11640.264296] Lustre: DEBUG MARKER: == sanity test 150e: Verify 60% of available OST space consumed by fallocate ========================================================== 16:56:33 (1734368193)
[11658.945806] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150f: Verify fallocate punch functionality ========================================================== 16:56:53 \(1734368213\)
[11659.292370] Lustre: DEBUG MARKER: == sanity test 150f: Verify fallocate punch functionality ========================================================== 16:56:53 (1734368213)
[11674.507926] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150g: Verify fallocate punch on large range ========================================================== 16:57:08 \(1734368228\)
[11674.945208] Lustre: DEBUG MARKER: == sanity test 150g: Verify fallocate punch on large range ========================================================== 16:57:08 (1734368228)
[11691.363327] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150h: Verify extend fallocate updates the file size ========================================================== 16:57:25 \(1734368245\)
[11691.784113] Lustre: DEBUG MARKER: == sanity test 150h: Verify extend fallocate updates the file size ========================================================== 16:57:25 (1734368245)
[11693.914037] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 151: test cache on oss and controls ========================================================================================= 16:57:28 \(1734368248\)
[11694.249300] Lustre: DEBUG MARKER: == sanity test 151: test cache on oss and controls ========================================================================================= 16:57:28 (1734368248)
[11708.626033] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 152: test read\/write with enomem ====================================================================================== 16:57:42 \(1734368262\)
[11708.962268] Lustre: DEBUG MARKER: == sanity test 152: test read/write with enomem ====================================================================================== 16:57:42 (1734368262)
[11711.043397] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 153: test if fdatasync does not crash ================================================================================= 16:57:45 \(1734368265\)
[11711.396383] Lustre: DEBUG MARKER: == sanity test 153: test if fdatasync does not crash ================================================================================= 16:57:45 (1734368265)
[11713.338689] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154A: lfs path2fid and fid2path basic checks ========================================================== 16:57:47 \(1734368267\)
[11713.687561] Lustre: DEBUG MARKER: == sanity test 154A: lfs path2fid and fid2path basic checks ========================================================== 16:57:47 (1734368267)
[11715.471920] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154B: verify the ll_decode_linkea tool ==== 16:57:49 \(1734368269\)
[11715.814956] Lustre: DEBUG MARKER: == sanity test 154B: verify the ll_decode_linkea tool ==== 16:57:49 (1734368269)
[11717.621357] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154a: Open-by-FID ========================= 16:57:51 \(1734368271\)
[11717.970238] Lustre: DEBUG MARKER: == sanity test 154a: Open-by-FID ========================= 16:57:51 (1734368271)
[11720.144763] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154b: Open-by-FID for remote directory ==== 16:57:54 \(1734368274\)
[11720.509610] Lustre: DEBUG MARKER: == sanity test 154b: Open-by-FID for remote directory ==== 16:57:54 (1734368274)
[11720.891950] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_154b needs \>= 2 MDTs
[11721.239816] Lustre: DEBUG MARKER: SKIP: sanity test_154b needs >= 2 MDTs
[11721.671880] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154c: lfs path2fid and fid2path multiple arguments ========================================================== 16:57:55 \(1734368275\)
[11722.025959] Lustre: DEBUG MARKER: == sanity test 154c: lfs path2fid and fid2path multiple arguments ========================================================== 16:57:55 (1734368275)
[11723.832978] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154d: Verify open file fid ================ 16:57:57 \(1734368277\)
[11724.176818] Lustre: DEBUG MARKER: == sanity test 154d: Verify open file fid ================ 16:57:57 (1734368277)
[11726.374498] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154e: .lustre is not returned by readdir == 16:58:00 \(1734368280\)
[11726.725653] Lustre: DEBUG MARKER: == sanity test 154e: .lustre is not returned by readdir == 16:58:00 (1734368280)
[11728.499025] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154ea: .lustre is not returned by readdir \(2\) ========================================================== 16:58:02 \(1734368282\)
[11728.843993] Lustre: DEBUG MARKER: == sanity test 154ea: .lustre is not returned by readdir (2) ========================================================== 16:58:02 (1734368282)
[11751.707789] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154f: get parent fids by reading link ea == 16:58:25 \(1734368305\)
[11752.138540] Lustre: DEBUG MARKER: == sanity test 154f: get parent fids by reading link ea == 16:58:25 (1734368305)
[11752.403457] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[11752.403457] [[ -z "" ]] && param= ||
[11752.403457] param=$(grep <<< "$params");
[11752.403457] [[ -z $param ]] && param="$params";
[11752.403457] while read s; do echo client $s;
[11752.403457] done <<< "$param"
[11752.479446] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff896191330800.xattr_cache=1
[11754.119094] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154g: various llapi FID tests ============= 16:58:28 \(1734368308\)
[11754.469160] Lustre: DEBUG MARKER: == sanity test 154g: various llapi FID tests ============= 16:58:28 (1734368308)
[11879.595284] Autotest: Test running for 195 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[11908.408183] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154h: Verify interactive path2fid ========= 17:01:02 \(1734368462\)
[11908.748388] Lustre: DEBUG MARKER: == sanity test 154h: Verify interactive path2fid ========= 17:01:02 (1734368462)
[11910.738628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154i: fid2path for path longer than PATH_MAX ========================================================== 17:01:04 \(1734368464\)
[11911.098704] Lustre: DEBUG MARKER: == sanity test 154i: fid2path for path longer than PATH_MAX ========================================================== 17:01:04 (1734368464)
[11916.685733] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155a: Verify small file correctness: read cache:on write_cache:on ========================================================== 17:01:10 \(1734368470\)
[11917.031126] Lustre: DEBUG MARKER: == sanity test 155a: Verify small file correctness: read cache:on write_cache:on ========================================================== 17:01:10 (1734368470)
[11924.957478] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155b: Verify small file correctness: read cache:on write_cache:off ========================================================== 17:01:19 \(1734368479\)
[11925.304897] Lustre: DEBUG MARKER: == sanity test 155b: Verify small file correctness: read cache:on write_cache:off ========================================================== 17:01:19 (1734368479)
[11933.299439] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155c: Verify small file correctness: read cache:off write_cache:on ========================================================== 17:01:27 \(1734368487\)
[11933.649774] Lustre: DEBUG MARKER: == sanity test 155c: Verify small file correctness: read cache:off write_cache:on ========================================================== 17:01:27 (1734368487)
[11941.577056] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155d: Verify small file correctness: read cache:off write_cache:off ========================================================== 17:01:35 \(1734368495\)
[11941.932868] Lustre: DEBUG MARKER: == sanity test 155d: Verify small file correctness: read cache:off write_cache:off ========================================================== 17:01:35 (1734368495)
[11950.956593] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155e: Verify big file correctness: read cache:on write_cache:on ========================================================== 17:01:45 \(1734368505\)
[11951.305009] Lustre: DEBUG MARKER: == sanity test 155e: Verify big file correctness: read cache:on write_cache:on ========================================================== 17:01:45 (1734368505)
[11963.029779] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155f: Verify big file correctness: read cache:on write_cache:off ========================================================== 17:01:57 \(1734368517\)
[11963.380744] Lustre: DEBUG MARKER: == sanity test 155f: Verify big file correctness: read cache:on write_cache:off ========================================================== 17:01:57 (1734368517)
[11974.976947] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155g: Verify big file correctness: read cache:off write_cache:on ========================================================== 17:02:09 \(1734368529\)
[11975.337544] Lustre: DEBUG MARKER: == sanity test 155g: Verify big file correctness: read cache:off write_cache:on ========================================================== 17:02:09 (1734368529)
[11992.000821] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155h: Verify big file correctness: read cache:off write_cache:off ========================================================== 17:02:26 \(1734368546\)
[11992.341951] Lustre: DEBUG MARKER: == sanity test 155h: Verify big file correctness: read cache:off write_cache:off ========================================================== 17:02:26 (1734368546)
[12003.914813] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 156: Verification of tunables ============= 17:02:38 \(1734368558\)
[12004.266402] Lustre: DEBUG MARKER: == sanity test 156: Verification of tunables ============= 17:02:38 (1734368558)
[12013.463749] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn on read and write cache
[12013.812453] Lustre: DEBUG MARKER: Turn on read and write cache
[12015.152595] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back.
[12015.496190] Lustre: DEBUG MARKER: Write data and read it back.
[12015.895767] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read should be satisfied from the cache.
[12016.242298] Lustre: DEBUG MARKER: Read should be satisfied from the cache.
[12017.611868] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits: before: 8277, after: 8280
[12017.966295] Lustre: DEBUG MARKER: cache hits: before: 8277, after: 8280
[12018.366171] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read again\; it should be satisfied from the cache.
[12018.713424] Lustre: DEBUG MARKER: Read again; it should be satisfied from the cache.
[12019.587249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8280, after: 8283
[12019.930508] Lustre: DEBUG MARKER: cache hits:: before: 8280, after: 8283
[12020.332709] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn off the read cache and turn on the write cache
[12020.693899] Lustre: DEBUG MARKER: Turn off the read cache and turn on the write cache
[12022.023232] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read again\; it should be satisfied from the cache.
[12022.365237] Lustre: DEBUG MARKER: Read again; it should be satisfied from the cache.
[12023.701891] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8283, after: 8286
[12024.041906] Lustre: DEBUG MARKER: cache hits:: before: 8283, after: 8286
[12024.432885] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back.
[12024.772892] Lustre: DEBUG MARKER: Write data and read it back.
[12025.172269] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read should be satisfied from the cache.
[12025.509323] Lustre: DEBUG MARKER: Read should be satisfied from the cache.
[12026.870006] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8286, after: 8289
[12027.212273] Lustre: DEBUG MARKER: cache hits:: before: 8286, after: 8289
[12027.618584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn off read and write cache
[12027.970562] Lustre: DEBUG MARKER: Turn off read and write cache
[12029.303081] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back
[12029.650508] Lustre: DEBUG MARKER: Write data and read it back
[12030.054135] Lustre: DEBUG MARKER: /usr/sbin/lctl mark It should not be satisfied from the cache.
[12030.392770] Lustre: DEBUG MARKER: It should not be satisfied from the cache.
[12032.162192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8289, after: 8289
[12032.506423] Lustre: DEBUG MARKER: cache hits:: before: 8289, after: 8289
[12032.888543] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn on the read cache and turn off the write cache
[12033.243848] Lustre: DEBUG MARKER: Turn on the read cache and turn off the write cache
[12034.572570] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back
[12034.943606] Lustre: DEBUG MARKER: Write data and read it back
[12035.329893] Lustre: DEBUG MARKER: /usr/sbin/lctl mark It should not be satisfied from the cache.
[12035.698351] Lustre: DEBUG MARKER: It should not be satisfied from the cache.
[12037.080557] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8289, after: 8289
[12037.428830] Lustre: DEBUG MARKER: cache hits:: before: 8289, after: 8289
[12037.836754] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read again\; it should be satisfied from the cache.
[12038.188870] Lustre: DEBUG MARKER: Read again; it should be satisfied from the cache.
[12039.536580] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8289, after: 8292
[12039.880224] Lustre: DEBUG MARKER: cache hits:: before: 8289, after: 8292
[12044.400435] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160a: changelog sanity ==================== 17:03:18 \(1734368598\)
[12044.751083] Lustre: DEBUG MARKER: == sanity test 160a: changelog sanity ==================== 17:03:18 (1734368598)
[12059.314502] LustreError: lustre-MDT0000-mdc-ffff896191330800: operation mds_set_info to node 10.240.38.186@tcp failed: rc = -107
[12059.316560] Lustre: lustre-MDT0000-mdc-ffff896191330800: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12059.324848] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff89618ddb0340 x1818615964717824/t30064776247(30064776247) o101->lustre-MDT0000-mdc-ffff896191330800@10.240.38.186@tcp:12/10 lens 984/608 e 0 to 0 dl 1734368629 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'rm.0' uid:0 gid:0
[12059.425367] Lustre: lustre-MDT0000-mdc-ffff896191330800: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[12059.527728] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[12059.532057] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f4d2d84a to 0x5d837553f4de457f
[12063.221384] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160b: Verify that very long rename doesn\'t crash in changelog ========================================================== 17:03:37 \(1734368617\)
[12063.567491] Lustre: DEBUG MARKER: == sanity test 160b: Verify that very long rename doesn't crash in changelog ========================================================== 17:03:37 (1734368617)
[12068.007584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160c: verify that changelog log catch the truncate event ========================================================== 17:03:42 \(1734368622\)
[12068.350974] Lustre: DEBUG MARKER: == sanity test 160c: verify that changelog log catch the truncate event ========================================================== 17:03:42 (1734368622)
[12071.498786] Lustre: 429912:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734368608/real 1734368608] req@ffff8962090e9d40 x1818615973966720/t0(0) o400->lustre-MDT0000-mdc-ffff896191330800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734368624 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[12073.237783] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160d: verify that changelog log catch the migrate event ========================================================== 17:03:47 \(1734368627\)
[12073.578334] Lustre: DEBUG MARKER: == sanity test 160d: verify that changelog log catch the migrate event ========================================================== 17:03:47 (1734368627)
[12074.006629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_160d needs \>= 2 MDTs
[12074.354658] Lustre: DEBUG MARKER: SKIP: sanity test_160d needs >= 2 MDTs
[12074.778997] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160e: changelog negative testing \(should return errors\) ========================================================== 17:03:48 \(1734368628\)
[12075.126809] Lustre: DEBUG MARKER: == sanity test 160e: changelog negative testing (should return errors) ========================================================== 17:03:48 (1734368628)
[12079.814876] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160f: changelog garbage collect \(timestamped users\) ========================================================== 17:03:53 \(1734368633\)
[12080.164280] Lustre: DEBUG MARKER: == sanity test 160f: changelog garbage collect (timestamped users) ========================================================== 17:03:53 (1734368633)
[12082.787449] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 1734368636: creating first files
[12083.134733] Lustre: DEBUG MARKER: 1734368636: creating first files
[12100.714226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160g: changelog garbage collect on idle records ========================================================== 17:04:14 \(1734368654\)
[12101.061117] Lustre: DEBUG MARKER: == sanity test 160g: changelog garbage collect on idle records ========================================================== 17:04:14 (1734368654)
[12114.193525] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160h: changelog gc thread stop upon umount, orphan records delete ========================================================== 17:04:28 \(1734368668\)
[12114.536160] Lustre: DEBUG MARKER: == sanity test 160h: changelog gc thread stop upon umount, orphan records delete ========================================================== 17:04:28 (1734368668)
[12141.449974] Lustre: lustre-MDT0000-mdc-ffff896191330800: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12141.457002] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[12141.460962] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f4de457f to 0x5d837553f4de508b
[12141.463381] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[12141.465022] Lustre: Skipped 1 previous similar message
[12141.466058] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff89618ddb0340 x1818615964717824/t30064776247(30064776247) o101->lustre-MDT0000-mdc-ffff896191330800@10.240.38.186@tcp:12/10 lens 984/608 e 0 to 0 dl 1734368711 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'rm.0' uid:0 gid:0
[12141.470624] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 77 previous similar messages
[12147.666001] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160i: changelog user register\/unregister race ========================================================== 17:05:01 \(1734368701\)
[12148.024052] Lustre: DEBUG MARKER: == sanity test 160i: changelog user register/unregister race ========================================================== 17:05:01 (1734368701)
[12152.327004] Lustre: 429913:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734368690/real 1734368690] req@ffff896192b189c0 x1818615974019968/t0(0) o400->lustre-MDT0000-mdc-ffff896191330800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734368706 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[12157.867925] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160j: client can be umounted while its chanangelog is being used ========================================================== 17:05:11 \(1734368711\)
[12158.187943] Lustre: DEBUG MARKER: == sanity test 160j: client can be umounted while its chanangelog is being used ========================================================== 17:05:11 (1734368711)
[12158.377916] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre2
[12158.389201] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre2
[12158.439187] Lustre: Mounted lustre-client
[12159.685197] Lustre: Unmounted lustre-client
[12161.400647] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[12161.412289] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[12161.460466] Lustre: Mounted lustre-client
[12162.959631] Lustre: Unmounted lustre-client
[12163.272584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160k: Verify that changelog records are not lost ========================================================== 17:05:17 \(1734368717\)
[12163.615126] Lustre: DEBUG MARKER: == sanity test 160k: Verify that changelog records are not lost ========================================================== 17:05:17 (1734368717)
[12175.307583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160l: Verify that MTIME changelog records contain the parent FID ========================================================== 17:05:29 \(1734368729\)
[12175.667175] Lustre: DEBUG MARKER: == sanity test 160l: Verify that MTIME changelog records contain the parent FID ========================================================== 17:05:29 (1734368729)
[12180.604375] Autotest: Test running for 200 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[12182.427231] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160m: Changelog clear race ================ 17:05:36 \(1734368736\)
[12182.769082] Lustre: DEBUG MARKER: == sanity test 160m: Changelog clear race ================ 17:05:36 (1734368736)
[12191.866192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160n: Changelog destroy race ============== 17:05:45 \(1734368745\)
[12192.200007] Lustre: DEBUG MARKER: == sanity test 160n: Changelog destroy race ============== 17:05:45 (1734368745)
[12194.385384] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12199.085525] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12256.401891] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12264.783548] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12266.625565] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12271.099250] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12328.274469] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12336.620279] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12338.466829] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12343.152635] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12401.044219] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12409.418878] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12417.546313] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160o: changelog user name and mask ======== 17:09:31 \(1734368971\)
[12417.898682] Lustre: DEBUG MARKER: == sanity test 160o: changelog user name and mask ======== 17:09:31 (1734368971)
[12428.250067] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160p: Changelog orphan cleanup with no users ========================================================== 17:09:42 \(1734368982\)
[12428.590299] Lustre: DEBUG MARKER: == sanity test 160p: Changelog orphan cleanup with no users ========================================================== 17:09:42 (1734368982)
[12437.896755] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[12437.899462] Lustre: lustre-MDT0000-mdc-ffff89618f22a800: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12437.902868] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f4de508b to 0x5d837553f521c68e
[12437.905293] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[12437.906919] Lustre: Skipped 1 previous similar message
[12437.916324] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff8961d51256c0 x1818615974104704/t38654705793(38654705793) o101->lustre-MDT0000-mdc-ffff89618f22a800@10.240.38.186@tcp:12/10 lens 576/608 e 0 to 0 dl 1734369008 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'lfs.0' uid:0 gid:0
[12437.920941] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 80 previous similar messages
[12440.504190] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160q: changelog effective mask is DEFMASK if not set ========================================================== 17:09:54 \(1734368994\)
[12440.857574] Lustre: DEBUG MARKER: == sanity test 160q: changelog effective mask is DEFMASK if not set ========================================================== 17:09:54 (1734368994)
[12444.145171] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160s: changelog garbage collect on idle records \* time ========================================================== 17:09:58 \(1734368998\)
[12444.490429] Lustre: DEBUG MARKER: == sanity test 160s: changelog garbage collect on idle records * time ========================================================== 17:09:58 (1734368998)
[12449.159760] Lustre: 429912:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734368987/real 1734368987] req@ffff8961dff20000 x1818616008756992/t0(0) o400->lustre-MDT0000-mdc-ffff89618f22a800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734369003 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[12456.747181] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160t: changelog garbage collect on lack of space ========================================================== 17:10:10 \(1734369010\)
[12457.096718] Lustre: DEBUG MARKER: == sanity test 160t: changelog garbage collect on lack of space ========================================================== 17:10:10 (1734369010)
[12458.936489] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12461.488928] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12464.459377] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12465.563924] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12476.640127] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160u: changelog rename record type name and sname strings are correct ========================================================== 17:10:30 \(1734369030\)
[12476.977062] Lustre: DEBUG MARKER: == sanity test 160u: changelog rename record type name and sname strings are correct ========================================================== 17:10:30 (1734369030)
[12481.215933] Autotest: Test running for 205 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[12485.390330] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161a: link ea sanity ====================== 17:10:39 \(1734369039\)
[12485.736688] Lustre: DEBUG MARKER: == sanity test 161a: link ea sanity ====================== 17:10:39 (1734369039)
[12486.577809] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12488.036221] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12490.907249] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12492.244796] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12493.456711] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161b: link ea sanity under remote directory ========================================================== 17:10:47 \(1734369047\)
[12493.803671] Lustre: DEBUG MARKER: == sanity test 161b: link ea sanity under remote directory ========================================================== 17:10:47 (1734369047)
[12494.203177] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_161b skipping remote directory test
[12494.540649] Lustre: DEBUG MARKER: SKIP: sanity test_161b skipping remote directory test
[12494.966778] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161c: check CL_RENME[UNLINK] changelog record flags ========================================================== 17:10:49 \(1734369049\)
[12495.304484] Lustre: DEBUG MARKER: == sanity test 161c: check CL_RENME[UNLINK] changelog record flags ========================================================== 17:10:49 (1734369049)
[12500.118368] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161d: create with concurrent .lustre\/fid access ========================================================== 17:10:54 \(1734369054\)
[12500.467562] Lustre: DEBUG MARKER: == sanity test 161d: create with concurrent .lustre/fid access ========================================================== 17:10:54 (1734369054)
[12501.756586] LustreError: 548976:0:(namei.c:1529:ll_create_node()) cfs_fail_timeout id 140c sleeping for 5000ms
[12503.839878] LustreError: 548976:0:(namei.c:1529:ll_create_node()) cfs_fail_timeout interrupted
[12506.594222] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 162a: path lookup sanity ================== 17:11:00 \(1734369060\)
[12506.939493] Lustre: DEBUG MARKER: == sanity test 162a: path lookup sanity ================== 17:11:00 (1734369060)
[12509.228531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 162b: striped directory path lookup sanity ========================================================== 17:11:03 \(1734369063\)
[12509.578051] Lustre: DEBUG MARKER: == sanity test 162b: striped directory path lookup sanity ========================================================== 17:11:03 (1734369063)
[12509.983198] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_162b needs \>= 2 MDTs
[12510.329286] Lustre: DEBUG MARKER: SKIP: sanity test_162b needs >= 2 MDTs
[12510.772126] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 162c: fid2path works with paths 100 or more directories deep ========================================================== 17:11:04 \(1734369064\)
[12511.123404] Lustre: DEBUG MARKER: == sanity test 162c: fid2path works with paths 100 or more directories deep ========================================================== 17:11:04 (1734369064)
[12517.264293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165a: ofd access log discovery ============ 17:11:11 \(1734369071\)
[12517.629653] Lustre: DEBUG MARKER: == sanity test 165a: ofd access log discovery ============ 17:11:11 (1734369071)
[12524.938450] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12540.168893] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165b: ofd access log entries are produced and consumed ========================================================== 17:11:34 \(1734369094\)
[12540.516658] Lustre: DEBUG MARKER: == sanity test 165b: ofd access log entries are produced and consumed ========================================================== 17:11:34 (1734369094)
[12565.897386] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12576.425715] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[12576.427624] Lustre: Skipped 1 previous similar message
[12578.380642] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165c: full ofd access logs do not block IOs ========================================================== 17:12:12 \(1734369132\)
[12578.730246] Lustre: DEBUG MARKER: == sanity test 165c: full ofd access logs do not block IOs ========================================================== 17:12:12 (1734369132)
[12586.282331] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12586.994798] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12593.552633] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12596.348287] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[12599.108422] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165d: ofd_access_log mask works =========== 17:12:33 \(1734369153\)
[12599.459807] Lustre: DEBUG MARKER: == sanity test 165d: ofd_access_log mask works =========== 17:12:33 (1734369153)
[12629.385977] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12631.961290] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[12634.605466] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165e: ofd_access_log MDT index filter works ========================================================== 17:13:08 \(1734369188\)
[12634.938125] Lustre: DEBUG MARKER: == sanity test 165e: ofd_access_log MDT index filter works ========================================================== 17:13:08 (1734369188)
[12635.348677] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_165e needs \>= 2 MDTs
[12635.692729] Lustre: DEBUG MARKER: SKIP: sanity test_165e needs >= 2 MDTs
[12636.114449] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165f: ofd_access_log_reader --exit-on-close works ========================================================== 17:13:10 \(1734369190\)
[12636.453076] Lustre: DEBUG MARKER: == sanity test 165f: ofd_access_log_reader --exit-on-close works ========================================================== 17:13:10 (1734369190)
[12652.584330] Lustre: lustre-OST0000-osc-ffff89618f22a800: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[12655.397670] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 169: parallel read and truncate should not deadlock ========================================================== 17:13:29 \(1734369209\)
[12655.749094] Lustre: DEBUG MARKER: == sanity test 169: parallel read and truncate should not deadlock ========================================================== 17:13:29 (1734369209)
[12656.152832] Lustre: DEBUG MARKER: /usr/sbin/lctl mark creating a 10 Mb file
[12656.499352] Lustre: DEBUG MARKER: creating a 10 Mb file
[12670.052167] Lustre: DEBUG MARKER: /usr/sbin/lctl mark starting reads
[12670.480210] Lustre: DEBUG MARKER: starting reads
[12670.948169] Lustre: DEBUG MARKER: /usr/sbin/lctl mark truncating the file
[12671.266307] Lustre: DEBUG MARKER: truncating the file
[12671.759343] Lustre: DEBUG MARKER: /usr/sbin/lctl mark killing dd
[12672.097022] Lustre: DEBUG MARKER: killing dd
[12672.496437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark removing the temporary file
[12672.840035] Lustre: DEBUG MARKER: removing the temporary file
[12674.632798] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 170: test lctl df to handle corrupted log =============================================================================== 17:13:48 \(1734369228\)
[12674.986129] Lustre: DEBUG MARKER: == sanity test 170: test lctl df to handle corrupted log =============================================================================== 17:13:48 (1734369228)
[12675.117002] Lustre: debug daemon will attempt to start writing to /tmp/f170.sanity_log_good (512000kB max)
[12675.130862] Lustre: shutting down debug daemon thread...
[12675.137754] Lustre: debug daemon will attempt to start writing to /tmp/f170.sanity_log_good (512000kB max)
[12675.149419] Lustre: shutting down debug daemon thread...
[12677.087418] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 171: test libcfs_debug_dumplog_thread stuck in do_exit\(\) ================================================================ 17:13:51 \(1734369231\)
[12677.425690] Lustre: DEBUG MARKER: == sanity test 171: test libcfs_debug_dumplog_thread stuck in do_exit() ================================================================ 17:13:51 (1734369231)
[12677.582392] LustreError: 558330:0:(file.c:457:ll_file_release()) cfs_fail_timeout id 50e sleeping for 3000ms
[12680.596342] LustreError: 558330:0:(file.c:457:ll_file_release()) cfs_fail_timeout id 50e awake
[12680.598023] LustreError: dumping log to /tmp/lustre-log.1734369234.558330
[12682.481229] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 172: manual device removal with lctl cleanup\/detach ================================================================ 17:13:56 \(1734369236\)
[12682.852716] Lustre: DEBUG MARKER: == sanity test 172: manual device removal with lctl cleanup/detach ================================================================ 17:13:56 (1734369236)
[12684.307328] Lustre: *** cfs_fail_loc=60e, val=0***
[12684.308369] Lustre: Unmounted lustre-client
[12685.709447] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[12685.720655] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[12685.773318] Lustre: Mounted lustre-client
[12686.091564] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 180a: test obdecho on osc ================= 17:14:00 \(1734369240\)
[12686.443165] Lustre: DEBUG MARKER: == sanity test 180a: test obdecho on osc ================= 17:14:00 (1734369240)
[12686.842159] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_180a obdecho on osc is no longer supported
[12687.183713] Lustre: DEBUG MARKER: SKIP: sanity test_180a obdecho on osc is no longer supported
[12687.610349] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 180b: test obdecho directly on obdfilter == 17:14:01 \(1734369241\)
[12687.951328] Lustre: DEBUG MARKER: == sanity test 180b: test obdecho directly on obdfilter == 17:14:01 (1734369241)
[12696.526086] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 180c: test huge bulk I\/O size on obdfilter, don\'t LASSERT ========================================================== 17:14:10 \(1734369250\)
[12696.873181] Lustre: DEBUG MARKER: == sanity test 180c: test huge bulk I/O size on obdfilter, don't LASSERT ========================================================== 17:14:10 (1734369250)
[12707.834972] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 181: Test open-unlinked dir ================================================================================== 17:14:21 \(1734369261\)
[12708.181688] Lustre: DEBUG MARKER: == sanity test 181: Test open-unlinked dir ================================================================================== 17:14:21 (1734369261)
[12708.897908] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12753.413860] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12755.094319] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12757.357596] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12760.156678] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 182a: Test parallel modify metadata operations from mdc ========================================================== 17:15:14 \(1734369314\)
[12760.502883] Lustre: DEBUG MARKER: == sanity test 182a: Test parallel modify metadata operations from mdc ========================================================== 17:15:14 (1734369314)
[12762.296681] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12762.369713] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12762.908164] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12763.055673] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12763.396250] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12763.459254] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12763.496251] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12763.943720] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12764.011580] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12764.075992] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12766.811456] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12767.036400] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12767.445622] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12767.499156] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12768.465452] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12768.529317] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12768.661278] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12769.024945] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12769.572484] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12770.566495] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12773.303961] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12773.479177] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12773.879861] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12773.909812] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12773.994413] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12774.199579] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12774.590715] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12774.613897] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12774.648720] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12774.661591] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12776.434205] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12776.684105] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12777.423561] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12777.916699] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12778.305842] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12778.377350] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12779.654067] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12779.824966] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12779.847771] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12780.043912] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12782.097946] Autotest: Test running for 210 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[12785.292913] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 17:15:39 \(1734369339\)
[12785.641635] Lustre: DEBUG MARKER: == sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 17:15:39 (1734369339)
[12786.035268] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_182b needs \>= 2 MDTs
[12786.376494] Lustre: DEBUG MARKER: SKIP: sanity test_182b needs >= 2 MDTs
[12786.814231] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 183: No crash or request leak in case of strange dispositions ================================================================== 17:15:40 \(1734369340\)
[12787.158757] Lustre: DEBUG MARKER: == sanity test 183: No crash or request leak in case of strange dispositions ================================================================== 17:15:40 (1734369340)
[12790.165842] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184a: Basic layout swap =================== 17:15:44 \(1734369344\)
[12790.512982] Lustre: DEBUG MARKER: == sanity test 184a: Basic layout swap =================== 17:15:44 (1734369344)
[12794.189412] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184b: Forbidden layout swap \(will generate errors\) ========================================================== 17:15:48 \(1734369348\)
[12794.523186] Lustre: DEBUG MARKER: == sanity test 184b: Forbidden layout swap (will generate errors) ========================================================== 17:15:48 (1734369348)
[12796.466887] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184c: Concurrent write and layout swap ==== 17:15:50 \(1734369350\)
[12796.816932] Lustre: DEBUG MARKER: == sanity test 184c: Concurrent write and layout swap ==== 17:15:50 (1734369350)
[12810.742457] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184d: allow stripeless layouts swap ======= 17:16:04 \(1734369364\)
[12811.091764] Lustre: DEBUG MARKER: == sanity test 184d: allow stripeless layouts swap ======= 17:16:04 (1734369364)
[12815.841343] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184e: Recreate layout after stripeless layout swaps ========================================================== 17:16:09 \(1734369369\)
[12816.180411] Lustre: DEBUG MARKER: == sanity test 184e: Recreate layout after stripeless layout swaps ========================================================== 17:16:09 (1734369369)
[12818.806276] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184f: IOC_MDC_GETFILEINFO for files with long names but no striping ========================================================== 17:16:12 \(1734369372\)
[12819.156559] Lustre: DEBUG MARKER: == sanity test 184f: IOC_MDC_GETFILEINFO for files with long names but no striping ========================================================== 17:16:12 (1734369372)
[12820.936822] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 185: Volatile file support ================ 17:16:15 \(1734369375\)
[12821.282505] Lustre: DEBUG MARKER: == sanity test 185: Volatile file support ================ 17:16:15 (1734369375)
[12824.134941] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 185a: Volatile file creation in .lustre\/fid\/ ========================================================== 17:16:18 \(1734369378\)
[12824.474296] Lustre: DEBUG MARKER: == sanity test 185a: Volatile file creation in .lustre/fid/ ========================================================== 17:16:18 (1734369378)
[12828.319260] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 187a: Test data version change ============ 17:16:22 \(1734369382\)
[12828.684777] Lustre: DEBUG MARKER: == sanity test 187a: Test data version change ============ 17:16:22 (1734369382)
[12833.120283] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 187b: Test data version change on volatile file ========================================================== 17:16:27 \(1734369387\)
[12833.467709] Lustre: DEBUG MARKER: == sanity test 187b: Test data version change on volatile file ========================================================== 17:16:27 (1734369387)
[12835.717567] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 200: OST pools ============================ 17:16:29 \(1734369389\)
[12836.060488] Lustre: DEBUG MARKER: == sanity test 200: OST pools ============================ 17:16:29 (1734369389)
[12838.566627] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1 2>/dev/null || echo foo
[12840.974162] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1 |
[12840.974162] grep -e lustre-OST0000_UUID -e lustre-OST0002_UUID -e lustre-OST0004_UUID -e lustre-OST0006_UUID | sort -u | tr '\n' ' '
[12844.873451] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1 | grep lustre-OST0000_UUID
[12852.367460] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1
[12856.188102] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1 2>/dev/null || echo foo
[12857.964429] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204a: Print default stripe attributes ===== 17:16:52 \(1734369412\)
[12858.324731] Lustre: DEBUG MARKER: == sanity test 204a: Print default stripe attributes ===== 17:16:52 (1734369412)
[12860.138087] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204b: Print default stripe size and offset ========================================================== 17:16:54 \(1734369414\)
[12860.486689] Lustre: DEBUG MARKER: == sanity test 204b: Print default stripe size and offset ========================================================== 17:16:54 (1734369414)
[12862.639179] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204c: Print default stripe count and offset ========================================================== 17:16:56 \(1734369416\)
[12862.991474] Lustre: DEBUG MARKER: == sanity test 204c: Print default stripe count and offset ========================================================== 17:16:56 (1734369416)
[12865.057308] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204d: Print default stripe count and size ========================================================== 17:16:59 \(1734369419\)
[12865.418350] Lustre: DEBUG MARKER: == sanity test 204d: Print default stripe count and size ========================================================== 17:16:59 (1734369419)
[12867.283802] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204e: Print raw stripe attributes ========= 17:17:01 \(1734369421\)
[12867.634136] Lustre: DEBUG MARKER: == sanity test 204e: Print raw stripe attributes ========= 17:17:01 (1734369421)
[12869.482978] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204f: Print raw stripe size and offset ==== 17:17:03 \(1734369423\)
[12869.830511] Lustre: DEBUG MARKER: == sanity test 204f: Print raw stripe size and offset ==== 17:17:03 (1734369423)
[12871.635631] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204g: Print raw stripe count and offset === 17:17:05 \(1734369425\)
[12871.999832] Lustre: DEBUG MARKER: == sanity test 204g: Print raw stripe count and offset === 17:17:05 (1734369425)
[12873.804747] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204h: Print raw stripe count and size ===== 17:17:07 \(1734369427\)
[12874.154205] Lustre: DEBUG MARKER: == sanity test 204h: Print raw stripe count and size ===== 17:17:07 (1734369427)
[12876.017630] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205a: Verify job stats ==================== 17:17:10 \(1734369430\)
[12876.372306] Lustre: DEBUG MARKER: == sanity test 205a: Verify job stats ==================== 17:17:10 (1734369430)
[12876.605833] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12876.624739] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12877.014719] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12878.033627] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12879.052137] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12881.175206] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/bin\/lfs mkdir -i 0 -c 1 \/mnt\/lustre\/d205a.sanity
[12881.518628] Lustre: DEBUG MARKER: Test: /usr/bin/lfs mkdir -i 0 -c 1 /mnt/lustre/d205a.sanity
[12881.920999] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.lfs.32710
[12882.266936] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.32710
[12883.035756] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: rmdir \/mnt\/lustre\/d205a.sanity
[12883.375891] Lustre: DEBUG MARKER: Test: rmdir /mnt/lustre/d205a.sanity
[12883.787391] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.rmdir.14341
[12884.133287] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.rmdir.14341
[12884.911924] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: mknod \/mnt\/lustre\/f205a.sanity c 1 3
[12885.259489] Lustre: DEBUG MARKER: Test: mknod /mnt/lustre/f205a.sanity c 1 3
[12885.666585] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.mknod.26626
[12886.011321] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.mknod.26626
[12886.786081] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: rm -f \/mnt\/lustre\/f205a.sanity
[12887.137619] Lustre: DEBUG MARKER: Test: rm -f /mnt/lustre/f205a.sanity
[12887.545560] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.rm.6494
[12887.894359] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.rm.6494
[12888.671095] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/bin\/lfs setstripe -i 0 -c 1 \/mnt\/lustre\/f205a.sanity
[12889.016687] Lustre: DEBUG MARKER: Test: /usr/bin/lfs setstripe -i 0 -c 1 /mnt/lustre/f205a.sanity
[12889.426726] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.lfs.22328
[12889.770069] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.22328
[12890.564570] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[12890.903873] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[12891.314989] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.touch.14910
[12891.684602] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.touch.14910
[12895.135626] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: dd if=\/dev\/zero of=\/mnt\/lustre\/f205a.sanity bs=1M count=1 oflag=sync
[12895.484126] Lustre: DEBUG MARKER: Test: dd if=/dev/zero of=/mnt/lustre/f205a.sanity bs=1M count=1 oflag=sync
[12895.891578] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.dd.32454
[12896.239058] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.dd.32454
[12897.577184] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: dd if=\/mnt\/lustre\/f205a.sanity of=\/dev\/null bs=1M count=1 iflag=direct
[12897.928866] Lustre: DEBUG MARKER: Test: dd if=/mnt/lustre/f205a.sanity of=/dev/null bs=1M count=1 iflag=direct
[12898.335610] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.dd.11651
[12898.679101] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.dd.11651
[12899.517629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/lib64\/lustre\/tests\/truncate \/mnt\/lustre\/f205a.sanity 0
[12899.871038] Lustre: DEBUG MARKER: Test: /usr/lib64/lustre/tests/truncate /mnt/lustre/f205a.sanity 0
[12900.289232] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.truncate.329
[12900.639815] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.truncate.329
[12901.805005] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: mv -f \/mnt\/lustre\/f205a.sanity \/mnt\/lustre\/d205a.sanity.rename
[12902.146367] Lustre: DEBUG MARKER: Test: mv -f /mnt/lustre/f205a.sanity /mnt/lustre/d205a.sanity.rename
[12902.557808] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.mv.30619
[12902.903337] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.mv.30619
[12903.707247] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/bin\/lfs mkdir -i 0 -c 1 \/mnt\/lustre\/d205a.sanity.expire
[12904.040679] Lustre: DEBUG MARKER: Test: /usr/bin/lfs mkdir -i 0 -c 1 /mnt/lustre/d205a.sanity.expire
[12904.457833] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.lfs.21760
[12904.796660] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.21760
[12905.744190] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12905.763430] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12906.151127] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12907.168566] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12907.540358] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[12907.900160] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[12908.328540] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment USER=S.root.touch.0.trevis-106vm14.t
[12908.670981] Lustre: DEBUG MARKER: Using JobID environment USER=S.root.touch.0.trevis-106vm14.t
[12909.465354] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[12909.803494] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[12910.209737] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment USER=S.root.touch.0.trevis-106vm14.E
[12910.544249] Lustre: DEBUG MARKER: Using JobID environment USER=S.root.touch.0.trevis-106vm14.E
[12911.333170] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[12911.675218] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[12912.106898] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment session=S.root.touch.0.trevis-106vm14.t
[12912.448201] Lustre: DEBUG MARKER: Using JobID environment session=S.root.touch.0.trevis-106vm14.t
[12915.814983] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12915.834979] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12916.223625] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12917.241407] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12918.261437] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[12918.576546] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205b: Verify job stats jobid and output format ========================================================== 17:17:52 \(1734369472\)
[12918.927857] Lustre: DEBUG MARKER: == sanity test 205b: Verify job stats jobid and output format ========================================================== 17:17:52 (1734369472)
[12935.976154] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205c: Verify client stats format ========== 17:18:10 \(1734369490\)
[12936.321220] Lustre: DEBUG MARKER: == sanity test 205c: Verify client stats format ========== 17:18:10 (1734369490)
[12938.108427] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205d: verify the format of some stats files ========================================================== 17:18:12 \(1734369492\)
[12938.474238] Lustre: DEBUG MARKER: == sanity test 205d: verify the format of some stats files ========================================================== 17:18:12 (1734369492)
[12942.443606] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205e: verify the output of lljobstat ====== 17:18:16 \(1734369496\)
[12942.796833] Lustre: DEBUG MARKER: == sanity test 205e: verify the output of lljobstat ====== 17:18:16 (1734369496)
[12954.501524] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205f: verify qos_ost_weights YAML format == 17:18:28 \(1734369508\)
[12954.862928] Lustre: DEBUG MARKER: == sanity test 205f: verify qos_ost_weights YAML format == 17:18:28 (1734369508)
[12957.406061] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205g: stress test for job_stats procfile == 17:18:31 \(1734369511\)
[12957.765514] Lustre: DEBUG MARKER: == sanity test 205g: stress test for job_stats procfile == 17:18:31 (1734369511)
[13051.702201] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205h: check jobid xattr is stored correctly ========================================================== 17:20:05 \(1734369605\)
[13052.051060] Lustre: DEBUG MARKER: == sanity test 205h: check jobid xattr is stored correctly ========================================================== 17:20:05 (1734369605)
[13060.365166] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205i: check job_xattr parameter accepts and rejects values correctly ========================================================== 17:20:14 \(1734369614\)
[13060.707138] Lustre: DEBUG MARKER: == sanity test 205i: check job_xattr parameter accepts and rejects values correctly ========================================================== 17:20:14 (1734369614)
[13065.264142] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205k: Verify \'?\' operator on job stats ==== 17:20:19 \(1734369619\)
[13065.617534] Lustre: DEBUG MARKER: == sanity test 205k: Verify '?' operator on job stats ==== 17:20:19 (1734369619)
[13069.682327] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 206: fail lov_init_raid0\(\) doesn\'t lbug === 17:20:23 \(1734369623\)
[13070.021199] Lustre: DEBUG MARKER: == sanity test 206: fail lov_init_raid0() doesn't lbug === 17:20:23 (1734369623)
[13070.170180] Lustre: *** cfs_fail_loc=1403, val=1***
[13071.807224] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 207a: can refresh layout at glimpse ======= 17:20:25 \(1734369625\)
[13072.152793] Lustre: DEBUG MARKER: == sanity test 207a: can refresh layout at glimpse ======= 17:20:25 (1734369625)
[13073.980378] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 207b: can refresh layout at open ========== 17:20:28 \(1734369628\)
[13074.319960] Lustre: DEBUG MARKER: == sanity test 207b: can refresh layout at open ========== 17:20:28 (1734369628)
[13076.138908] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 208: Exclusive open ======================= 17:20:30 \(1734369630\)
[13076.482049] Lustre: DEBUG MARKER: == sanity test 208: Exclusive open ======================= 17:20:30 (1734369630)
[13084.732208] Autotest: Test running for 215 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[13100.426731] Lustre: lustre-MDT0000-mdc-ffff89618e2aa800: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[13100.429701] Lustre: Skipped 1 previous similar message
[13100.430995] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[13100.436580] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f5250af2 to 0x5d837553f535f9c0
[13100.438992] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[13100.441412] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff896227d98680 x1818616026333824/t42949738492(42949738492) o101->lustre-MDT0000-mdc-ffff89618e2aa800@10.240.38.186@tcp:12/10 lens 576/608 e 0 to 0 dl 1734369739 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'multiop.0' uid:0 gid:0
[13102.427167] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[13103.549740] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[13103.915833] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[13104.229409] Lustre: DEBUG MARKER: lctl get_param -n at_max
[13104.483030] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[13104.939114] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[13126.027953] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[13126.031253] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f535f9c0 to 0x5d837553f535fd0f
[13126.039045] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff896227d9aa40 x1818616026344960/t47244640259(47244640259) o101->lustre-MDT0000-mdc-ffff89618e2aa800@10.240.38.186@tcp:12/10 lens 584/608 e 0 to 0 dl 1734369765 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'multiop.0' uid:0 gid:0
[13126.043738] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 1 previous similar message
[13127.907779] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[13129.070643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[13129.443791] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[13129.734734] Lustre: DEBUG MARKER: lctl get_param -n at_max
[13129.988705] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[13130.438307] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[13134.045575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 209: read-only open\/close requests should be freed promptly ========================================================== 17:21:28 \(1734369688\)
[13134.394294] Lustre: DEBUG MARKER: == sanity test 209: read-only open/close requests should be freed promptly ========================================================== 17:21:28 (1734369688)
[13147.413450] bash (603344): drop_caches: 3
[13149.570529] bash (603344): drop_caches: 3
[13151.683514] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 210: lfs getstripe does not break leases == 17:21:45 \(1734369705\)
[13152.026958] Lustre: DEBUG MARKER: == sanity test 210: lfs getstripe does not break leases == 17:21:45 (1734369705)
[13155.906673] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 211: failed mirror split doesn\'t break write lease ========================================================== 17:21:49 \(1734369709\)
[13156.245013] Lustre: DEBUG MARKER: == sanity test 211: failed mirror split doesn't break write lease ========================================================== 17:21:49 (1734369709)
[13171.081669] Lustre: 429913:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734369639/real 1734369639] req@ffff896227d9ad80 x1818616026334464/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2aa800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734369725 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[13176.201761] Lustre: 429913:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734369644/real 1734369644] req@ffff896227d9a700 x1818616026335616/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2aa800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734369730 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[13180.841699] Lustre: 429914:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734369649/real 1734369649] req@ffff896227d99040 x1818616026336768/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2aa800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734369735 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[13196.425751] Lustre: 429911:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734369665/real 1734369665] req@ffff89620ea5fa80 x1818616026345856/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2aa800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734369750 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[13200.937749] Lustre: 429913:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734369670/real 1734369670] req@ffff896227d989c0 x1818616026347008/t0(0) o400->lustre-MDT0000-mdc-ffff89618e2aa800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734369755 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[13246.849613] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 212: Sendfile test ====================================================================================================== 17:23:20 \(1734369800\)
[13247.202904] Lustre: DEBUG MARKER: == sanity test 212: Sendfile test ====================================================================================================== 17:23:20 (1734369800)
[13249.154766] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 213: OSC lock completion and cancel race don\'t crash - bug 18829 ========================================================== 17:23:23 \(1734369803\)
[13249.500859] Lustre: DEBUG MARKER: == sanity test 213: OSC lock completion and cancel race don't crash - bug 18829 ========================================================== 17:23:23 (1734369803)
[13261.177704] LustreError: 429913:0:(osc_request.c:3109:osc_enqueue_interpret()) cfs_fail_timeout id 40f sleeping for 10000ms
[13271.257950] LustreError: 429913:0:(osc_request.c:3109:osc_enqueue_interpret()) cfs_fail_timeout id 40f awake
[13273.008575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 214: hash-indexed directory test - bug 20133 ========================================================== 17:23:47 \(1734369827\)
[13273.358615] Lustre: DEBUG MARKER: == sanity test 214: hash-indexed directory test - bug 20133 ========================================================== 17:23:47 (1734369827)
[13285.711460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 215: lnet exists and has proper content - bugs 18102, 21079, 21517 ========================================================== 17:23:59 \(1734369839\)
[13286.060016] Lustre: DEBUG MARKER: == sanity test 215: lnet exists and has proper content - bugs 18102, 21079, 21517 ========================================================== 17:23:59 (1734369839)
[13288.015888] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 216: check lockless direct write updates file size and kms correctly ========================================================== 17:24:02 \(1734369842\)
[13288.344317] Lustre: DEBUG MARKER: == sanity test 216: check lockless direct write updates file size and kms correctly ========================================================== 17:24:02 (1734369842)
[13288.667427] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param osc.*.contention_seconds);
[13288.667427] [[ -z "" ]] && param= ||
[13288.667427] param=$(grep <<< "$params");
[13288.667427] [[ -z $param ]] && param="$params";
[13288.667427] while read s; do echo client $s;
[13288.667427] done <<< "$param"
[13297.549521] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n =
[13306.980490] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 217: check lctl ping for hostnames with embedded hyphen \(\'-\'\) ========================================================== 17:24:21 \(1734369861\)
[13307.318117] Lustre: DEBUG MARKER: == sanity test 217: check lctl ping for hostnames with embedded hyphen ('-') ========================================================== 17:24:21 (1734369861)
[13307.531116] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[13307.548054] Lustre: DEBUG MARKER: getent ahostsv4 trevis-106vm14.trevis.whamcloud.com
[13313.562217] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 218: parallel read and truncate should not deadlock ========================================================== 17:24:27 \(1734369867\)
[13313.903496] Lustre: DEBUG MARKER: == sanity test 218: parallel read and truncate should not deadlock ========================================================== 17:24:27 (1734369867)
[13314.330195] Lustre: DEBUG MARKER: /usr/sbin/lctl mark creating a 10 Mb file
[13314.651181] Lustre: DEBUG MARKER: creating a 10 Mb file
[13326.695218] Lustre: DEBUG MARKER: /usr/sbin/lctl mark starting reads
[13327.103015] Lustre: DEBUG MARKER: starting reads
[13327.593287] Lustre: DEBUG MARKER: /usr/sbin/lctl mark truncating the file
[13327.894895] Lustre: DEBUG MARKER: truncating the file
[13328.411613] Lustre: DEBUG MARKER: /usr/sbin/lctl mark killing dd
[13328.744768] Lustre: DEBUG MARKER: killing dd
[13329.163300] Lustre: DEBUG MARKER: /usr/sbin/lctl mark removing the temporary file
[13329.499541] Lustre: DEBUG MARKER: removing the temporary file
[13331.317535] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 219: LU-394: Write partial won\'t cause uncontiguous pages vec at LND ========================================================== 17:24:45 \(1734369885\)
[13331.666730] Lustre: DEBUG MARKER: == sanity test 219: LU-394: Write partial won't cause uncontiguous pages vec at LND ========================================================== 17:24:45 (1734369885)
[13331.819491] Lustre: *** cfs_fail_loc=411, val=0***
[13333.999926] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 220: preallocated MDS objects still used if ENOSPC from OST ========================================================== 17:24:48 \(1734369888\)
[13334.372933] Lustre: DEBUG MARKER: == sanity test 220: preallocated MDS objects still used if ENOSPC from OST ========================================================== 17:24:48 (1734369888)
[13338.489201] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_220 2>/dev/null || echo foo
[13349.432332] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 221: make sure fault and truncate race to not cause OOM ========================================================== 17:25:03 \(1734369903\)
[13349.788197] Lustre: DEBUG MARKER: == sanity test 221: make sure fault and truncate race to not cause OOM ========================================================== 17:25:03 (1734369903)
[13379.685123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 222a: AGL for ls should not trigger CLIO lock failure ========================================================== 17:25:33 \(1734369933\)
[13380.035843] Lustre: DEBUG MARKER: == sanity test 222a: AGL for ls should not trigger CLIO lock failure ========================================================== 17:25:33 (1734369933)
[13381.971409] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 222b: AGL for rmdir should not trigger CLIO lock failure ========================================================== 17:25:36 \(1734369936\)
[13382.317286] Lustre: DEBUG MARKER: == sanity test 222b: AGL for rmdir should not trigger CLIO lock failure ========================================================== 17:25:36 (1734369936)
[13384.239657] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 223: osc reenqueue if without AGL lock granted ================================================================================= 17:25:38 \(1734369938\)
[13384.572580] Lustre: DEBUG MARKER: == sanity test 223: osc reenqueue if without AGL lock granted ================================================================================= 17:25:38 (1734369938)
[13386.498452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224a: Don\'t panic on bulk IO failure ====== 17:25:40 \(1734369940\)
[13386.915088] Lustre: DEBUG MARKER: == sanity test 224a: Don't panic on bulk IO failure ====== 17:25:40 (1734369940)
[13387.038045] Lustre: *** cfs_fail_loc=508, val=2147483648***
[13387.039242] LustreError: 429906:0:(events.c:209:client_bulk_callback()) event type 1, status -5, desc ffff8961d4cfb400
[13387.042440] Lustre: 429911:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1734369941/real 1734369941] req@ffff8961d2dc4000 x1818616027062528/t0(0) o4->lustre-OST0000-osc-ffff89618e2aa800@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 1 dl 1734369996 ref 2 fl Rpc:eXQr/200/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0
[13387.047559] Lustre: 429911:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[13387.049199] Lustre: lustre-OST0000-osc-ffff89618e2aa800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[13387.052061] Lustre: Skipped 1 previous similar message
[13387.357343] Autotest: Test running for 220 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[13445.514418] Lustre: 429911:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734369944/real 1734369944] req@ffff8961d2dc4000 x1818616027062528/t47244641838(47244641838) o4->lustre-OST0000-osc-ffff89618e2aa800@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 1 dl 1734369999 ref 2 fl Bulk:RXMQU/204/0 rc 0/0 job:'dd.0' uid:0 gid:0
[13445.519794] LustreError: 429911:0:(client.c:2233:ptlrpc_check_set()) @@@ bulk transfer failed 0/1048576/0 req@ffff8961d2dc4000 x1818616027062528/t47244641838(47244641838) o4->lustre-OST0000-osc-ffff89618e2aa800@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 1 dl 1734369999 ref 2 fl Bulk:ReXMQU/204/0 rc 0/0 job:'dd.0' uid:0 gid:0
[13445.524395] LustreError: 429911:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961d2dc4000 x1818616027062528/t47244641838(47244641838) o4->lustre-OST0000-osc-ffff89618e2aa800@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 1 dl 1734369999 ref 2 fl Interpret:ReXMQU/204/0 rc -5/0 job:'dd.0' uid:0 gid:0
[13445.529572] Lustre: lustre-OST0000-osc-ffff89618e2aa800: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[13445.531429] Lustre: Skipped 3 previous similar messages
[13466.592189] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224b: Don\'t panic on bulk IO failure ====== 17:27:00 \(1734370020\)
[13466.940093] Lustre: DEBUG MARKER: == sanity test 224b: Don't panic on bulk IO failure ====== 17:27:00 (1734370020)
[13468.508305] Lustre: DEBUG MARKER: lctl set_param at_max=0
[13474.210879] Lustre: DEBUG MARKER: lctl set_param at_max=600
[13474.784977] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224c: Don\'t hang if one of md lost during large bulk RPC ========================================================== 17:27:08 \(1734370028\)
[13475.115200] Lustre: DEBUG MARKER: == sanity test 224c: Don't hang if one of md lost during large bulk RPC ========================================================== 17:27:08 (1734370028)
[13478.448430] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13478.464472] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13478.859311] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13479.876344] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13480.893356] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13480.911328] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13480.928594] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13481.319427] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13482.336452] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13483.353410] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13488.970519] Lustre: 429914:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734370038/real 1734370038] req@ffff896228143400 x1818616027082880/t0(0) o4->lustre-OST0000-osc-ffff89618e2aa800@10.240.45.50@tcp:6/4 lens 488/448 e 0 to 1 dl 1734370043 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0
[13489.757598] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13489.774537] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13490.159816] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13491.176826] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13492.194059] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[13492.211294] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13492.227663] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13492.614364] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13493.631822] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13494.648875] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[13499.008679] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224d: Don\'t corrupt data on bulk IO timeout ========================================================== 17:27:33 \(1734370053\)
[13499.346517] Lustre: DEBUG MARKER: == sanity test 224d: Don't corrupt data on bulk IO timeout ========================================================== 17:27:33 (1734370053)
[13500.561450] Lustre: DEBUG MARKER: lctl set_param at_max=0
[13520.778688] Lustre: lustre-OST0000-osc-ffff89618e2aa800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[13520.781527] Lustre: Skipped 2 previous similar messages
[13520.782471] LustreError: 429911:0:(client.c:2233:ptlrpc_check_set()) @@@ bulk transfer failed 0/1048576/0 req@ffff8961d2dc5d40 x1818616027094144/t0(0) o3->lustre-OST0000-osc-ffff89618e2aa800@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 1 dl 1734370075 ref 2 fl Bulk:ReXMQU/200/0 rc 0/0 job:'dd.0' uid:0 gid:0
[13520.786674] LustreError: 429911:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961d2dc5d40 x1818616027094144/t0(0) o3->lustre-OST0000-osc-ffff89618e2aa800@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 1 dl 1734370075 ref 2 fl Interpret:ReXMQU/200/0 rc -5/0 job:'dd.0' uid:0 gid:0
[13524.003299] Lustre: DEBUG MARKER: lctl set_param at_max=600
[13524.542561] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 225a: Metadata survey sanity with zero-stripe ========================================================== 17:27:58 \(1734370078\)
[13524.891178] Lustre: DEBUG MARKER: == sanity test 225a: Metadata survey sanity with zero-stripe ========================================================== 17:27:58 (1734370078)
[13531.060455] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 225b: Metadata survey sanity with stripe_count = 1 ========================================================== 17:28:05 \(1734370085\)
[13531.405844] Lustre: DEBUG MARKER: == sanity test 225b: Metadata survey sanity with stripe_count = 1 ========================================================== 17:28:05 (1734370085)
[13537.682525] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226a: call path2fid and fid2path on files of all type ========================================================== 17:28:11 \(1734370091\)
[13538.061529] Lustre: DEBUG MARKER: == sanity test 226a: call path2fid and fid2path on files of all type ========================================================== 17:28:11 (1734370091)
[13539.966583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226b: call path2fid and fid2path on files of all type under remote dir ========================================================== 17:28:14 \(1734370094\)
[13540.304771] Lustre: DEBUG MARKER: == sanity test 226b: call path2fid and fid2path on files of all type under remote dir ========================================================== 17:28:14 (1734370094)
[13540.701364] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_226b needs \>= 2 MDTs
[13541.038056] Lustre: DEBUG MARKER: SKIP: sanity test_226b needs >= 2 MDTs
[13541.496410] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226c: call path2fid and fid2path under remote dir with subdir mount ========================================================== 17:28:15 \(1734370095\)
[13541.845972] Lustre: DEBUG MARKER: == sanity test 226c: call path2fid and fid2path under remote dir with subdir mount ========================================================== 17:28:15 (1734370095)
[13542.249748] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_226c needs \>= 2 MDTs
[13542.595028] Lustre: DEBUG MARKER: SKIP: sanity test_226c needs >= 2 MDTs
[13543.035583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226d: verify fid2path with -n and -fn option ========================================================== 17:28:17 \(1734370097\)
[13543.405745] Lustre: DEBUG MARKER: == sanity test 226d: verify fid2path with -n and -fn option ========================================================== 17:28:17 (1734370097)
[13549.794645] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226e: Verify path2fid -0 option with newline and space ========================================================== 17:28:23 \(1734370103\)
[13550.164141] Lustre: DEBUG MARKER: == sanity test 226e: Verify path2fid -0 option with newline and space ========================================================== 17:28:23 (1734370103)
[13552.018407] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 227: running truncated executable does not cause OOM ========================================================== 17:28:26 \(1734370106\)
[13552.371230] Lustre: DEBUG MARKER: == sanity test 227: running truncated executable does not cause OOM ========================================================== 17:28:26 (1734370106)
[13557.740084] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 228a: try to reuse idle OI blocks ========= 17:28:31 \(1734370111\)
[13558.074410] Lustre: DEBUG MARKER: == sanity test 228a: try to reuse idle OI blocks ========= 17:28:31 (1734370111)
[13558.846635] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13559.314351] Lustre: *** cfs_fail_loc=1002, val=0***
[13568.092968] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13577.027038] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13581.741074] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13583.260747] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13586.176160] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13595.543933] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 228b: idle OI blocks can be reused after MDT restart ========================================================== 17:29:09 \(1734370149\)
[13595.861289] Lustre: DEBUG MARKER: == sanity test 228b: idle OI blocks can be reused after MDT restart ========================================================== 17:29:09 (1734370149)
[13596.615335] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13597.092308] Lustre: *** cfs_fail_loc=1002, val=0***
[13605.446734] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13617.340681] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13622.098234] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13628.300293] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[13628.458801] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f535fd0f to 0x5d837553f54b08c3
[13628.461134] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[13628.462761] Lustre: Skipped 2 previous similar messages
[13633.938766] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13636.198100] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13645.649901] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 228c: NOT shrink the last entry in OI index node to recycle idle leaf ========================================================== 17:29:59 \(1734370199\)
[13645.975835] Lustre: DEBUG MARKER: == sanity test 228c: NOT shrink the last entry in OI index node to recycle idle leaf ========================================================== 17:29:59 (1734370199)
[13646.728660] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13647.196644] Lustre: *** cfs_fail_loc=1002, val=0***
[13663.640118] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13675.808062] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13684.689978] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13686.195234] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13688.273573] Autotest: Test running for 225 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[13688.703919] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13698.302504] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 229: getstripe\/stat\/rm\/attr changes work on released files ========================================================== 17:30:52 \(1734370252\)
[13698.648970] Lustre: DEBUG MARKER: == sanity test 229: getstripe/stat/rm/attr changes work on released files ========================================================== 17:30:52 (1734370252)
[13700.502006] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230a: Create remote directory and files under the remote directory ========================================================== 17:30:54 \(1734370254\)
[13700.855127] Lustre: DEBUG MARKER: == sanity test 230a: Create remote directory and files under the remote directory ========================================================== 17:30:54 (1734370254)
[13701.268358] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230a needs \>= 2 MDTs
[13701.636352] Lustre: DEBUG MARKER: SKIP: sanity test_230a needs >= 2 MDTs
[13702.077828] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230b: migrate directory =================== 17:30:56 \(1734370256\)
[13702.404430] Lustre: DEBUG MARKER: == sanity test 230b: migrate directory =================== 17:30:56 (1734370256)
[13702.800716] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230b needs \>= 2 MDTs
[13703.153530] Lustre: DEBUG MARKER: SKIP: sanity test_230b needs >= 2 MDTs
[13703.570676] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230c: check directory accessiblity if migration failed ========================================================== 17:30:57 \(1734370257\)
[13703.917341] Lustre: DEBUG MARKER: == sanity test 230c: check directory accessiblity if migration failed ========================================================== 17:30:57 (1734370257)
[13704.316643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230c needs \>= 2 MDTs
[13704.661538] Lustre: DEBUG MARKER: SKIP: sanity test_230c needs >= 2 MDTs
[13705.088826] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230d skipping SLOW test 230d
[13705.420611] Lustre: DEBUG MARKER: SKIP: sanity test_230d skipping SLOW test 230d
[13705.844161] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230e: migrate mulitple local link files === 17:30:59 \(1734370259\)
[13706.185344] Lustre: DEBUG MARKER: == sanity test 230e: migrate mulitple local link files === 17:30:59 (1734370259)
[13706.583085] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230e needs \>= 2 MDTs
[13706.924545] Lustre: DEBUG MARKER: SKIP: sanity test_230e needs >= 2 MDTs
[13707.363979] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230f: migrate mulitple remote link files == 17:31:01 \(1734370261\)
[13707.707155] Lustre: DEBUG MARKER: == sanity test 230f: migrate mulitple remote link files == 17:31:01 (1734370261)
[13708.108372] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230f needs \>= 2 MDTs
[13708.475198] Lustre: DEBUG MARKER: SKIP: sanity test_230f needs >= 2 MDTs
[13708.929917] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230g: migrate dir to non-exist MDT ======== 17:31:02 \(1734370262\)
[13709.256768] Lustre: DEBUG MARKER: == sanity test 230g: migrate dir to non-exist MDT ======== 17:31:02 (1734370262)
[13709.637359] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230g needs \>= 2 MDTs
[13709.980621] Lustre: DEBUG MARKER: SKIP: sanity test_230g needs >= 2 MDTs
[13710.414413] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230h: migrate .. and root ================= 17:31:04 \(1734370264\)
[13710.756952] Lustre: DEBUG MARKER: == sanity test 230h: migrate .. and root ================= 17:31:04 (1734370264)
[13711.154955] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230h needs \>= 2 MDTs
[13711.497749] Lustre: DEBUG MARKER: SKIP: sanity test_230h needs >= 2 MDTs
[13711.939732] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230i: lfs migrate -m tolerates trailing slashes ========================================================== 17:31:06 \(1734370266\)
[13712.278252] Lustre: DEBUG MARKER: == sanity test 230i: lfs migrate -m tolerates trailing slashes ========================================================== 17:31:06 (1734370266)
[13712.684450] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230i needs \>= 2 MDTs
[13713.051005] Lustre: DEBUG MARKER: SKIP: sanity test_230i needs >= 2 MDTs
[13713.477643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230j: DoM file data not changed after dir migration ========================================================== 17:31:07 \(1734370267\)
[13713.836587] Lustre: DEBUG MARKER: == sanity test 230j: DoM file data not changed after dir migration ========================================================== 17:31:07 (1734370267)
[13714.253387] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230j needs \>= 2 MDTs
[13714.599870] Lustre: DEBUG MARKER: SKIP: sanity test_230j needs >= 2 MDTs
[13715.054393] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230k: file data not changed after dir migration ========================================================== 17:31:09 \(1734370269\)
[13715.399944] Lustre: DEBUG MARKER: == sanity test 230k: file data not changed after dir migration ========================================================== 17:31:09 (1734370269)
[13715.803517] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230k needs \>= 4 MDTs
[13716.151269] Lustre: DEBUG MARKER: SKIP: sanity test_230k needs >= 4 MDTs
[13716.589158] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230l: readdir between MDTs won\'t crash ==== 17:31:10 \(1734370270\)
[13716.937038] Lustre: DEBUG MARKER: == sanity test 230l: readdir between MDTs won't crash ==== 17:31:10 (1734370270)
[13717.342174] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230l needs \>= 2 MDTs
[13717.685242] Lustre: DEBUG MARKER: SKIP: sanity test_230l needs >= 2 MDTs
[13718.118195] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230m: xattrs not changed after dir migration ========================================================== 17:31:12 \(1734370272\)
[13718.473253] Lustre: DEBUG MARKER: == sanity test 230m: xattrs not changed after dir migration ========================================================== 17:31:12 (1734370272)
[13718.875694] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230m needs \>= 2 MDTs
[13719.212616] Lustre: DEBUG MARKER: SKIP: sanity test_230m needs >= 2 MDTs
[13719.664612] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230n: Dir migration with mirrored file ==== 17:31:13 \(1734370273\)
[13719.988746] Lustre: DEBUG MARKER: == sanity test 230n: Dir migration with mirrored file ==== 17:31:13 (1734370273)
[13720.419131] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230n needs \>= 2 MDTs
[13720.731977] Lustre: DEBUG MARKER: SKIP: sanity test_230n needs >= 2 MDTs
[13721.167585] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230o: dir split =========================== 17:31:15 \(1734370275\)
[13721.535716] Lustre: DEBUG MARKER: == sanity test 230o: dir split =========================== 17:31:15 (1734370275)
[13721.921437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230o needs \>= 2 MDTs
[13722.281494] Lustre: DEBUG MARKER: SKIP: sanity test_230o needs >= 2 MDTs
[13722.716826] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230p: dir merge =========================== 17:31:16 \(1734370276\)
[13723.066059] Lustre: DEBUG MARKER: == sanity test 230p: dir merge =========================== 17:31:16 (1734370276)
[13723.467335] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230p needs \>= 2 MDTs
[13723.809077] Lustre: DEBUG MARKER: SKIP: sanity test_230p needs >= 2 MDTs
[13724.245160] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230q: dir auto split ====================== 17:31:18 \(1734370278\)
[13724.611871] Lustre: DEBUG MARKER: == sanity test 230q: dir auto split ====================== 17:31:18 (1734370278)
[13724.993105] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230q needs \>= 2 MDTs
[13725.360371] Lustre: DEBUG MARKER: SKIP: sanity test_230q needs >= 2 MDTs
[13725.787293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230r: migrate with too many local locks === 17:31:19 \(1734370279\)
[13726.136899] Lustre: DEBUG MARKER: == sanity test 230r: migrate with too many local locks === 17:31:19 (1734370279)
[13726.556306] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230r needs \>= 2 MDTs
[13726.888059] Lustre: DEBUG MARKER: SKIP: sanity test_230r needs >= 2 MDTs
[13727.340798] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230s: lfs mkdir should return -EEXIST if target exists ========================================================== 17:31:21 \(1734370281\)
[13727.695744] Lustre: DEBUG MARKER: == sanity test 230s: lfs mkdir should return -EEXIST if target exists ========================================================== 17:31:21 (1734370281)
[13731.198201] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230t: migrate directory with project ID set ========================================================== 17:31:25 \(1734370285\)
[13731.546422] Lustre: DEBUG MARKER: == sanity test 230t: migrate directory with project ID set ========================================================== 17:31:25 (1734370285)
[13731.970173] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230t needs \>= 2 MDTs
[13732.308263] Lustre: DEBUG MARKER: SKIP: sanity test_230t needs >= 2 MDTs
[13732.754198] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230u: migrate directory by QOS ============ 17:31:26 \(1734370286\)
[13733.105287] Lustre: DEBUG MARKER: == sanity test 230u: migrate directory by QOS ============ 17:31:26 (1734370286)
[13733.513782] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230u needs \>= 4 MDTs
[13733.882719] Lustre: DEBUG MARKER: SKIP: sanity test_230u needs >= 4 MDTs
[13734.288596] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230v: subdir migrated to the MDT where its parent is located ========================================================== 17:31:28 \(1734370288\)
[13734.634636] Lustre: DEBUG MARKER: == sanity test 230v: subdir migrated to the MDT where its parent is located ========================================================== 17:31:28 (1734370288)
[13735.036836] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230v needs \>= 4 MDTs
[13735.397264] Lustre: DEBUG MARKER: SKIP: sanity test_230v needs >= 4 MDTs
[13735.853960] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230w: non-recursive mode dir migration ==== 17:31:29 \(1734370289\)
[13736.191986] Lustre: DEBUG MARKER: == sanity test 230w: non-recursive mode dir migration ==== 17:31:29 (1734370289)
[13736.597939] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230w needs \>= 2 MDTs
[13736.961767] Lustre: DEBUG MARKER: SKIP: sanity test_230w needs >= 2 MDTs
[13737.392975] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230x: dir migration check space =========== 17:31:31 \(1734370291\)
[13737.766212] Lustre: DEBUG MARKER: == sanity test 230x: dir migration check space =========== 17:31:31 (1734370291)
[13738.175099] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230x needs \>= 2 MDTs
[13738.539692] Lustre: DEBUG MARKER: SKIP: sanity test_230x needs >= 2 MDTs
[13739.031437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230y: unlink dir with bad hash type ======= 17:31:33 \(1734370293\)
[13739.383966] Lustre: DEBUG MARKER: == sanity test 230y: unlink dir with bad hash type ======= 17:31:33 (1734370293)
[13739.790130] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230y needs \>= 2 MDTs
[13740.136726] Lustre: DEBUG MARKER: SKIP: sanity test_230y needs >= 2 MDTs
[13740.646116] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230z: resume dir migration with bad hash type ========================================================== 17:31:34 \(1734370294\)
[13740.986390] Lustre: DEBUG MARKER: == sanity test 230z: resume dir migration with bad hash type ========================================================== 17:31:34 (1734370294)
[13741.426750] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_230z needs \>= 2 MDTs
[13741.778342] Lustre: DEBUG MARKER: SKIP: sanity test_230z needs >= 2 MDTs
[13742.196048] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 231a: checking that reading\/writing of BRW RPC size results in one RPC ========================================================== 17:31:36 \(1734370296\)
[13742.577518] Lustre: DEBUG MARKER: == sanity test 231a: checking that reading/writing of BRW RPC size results in one RPC ========================================================== 17:31:36 (1734370296)
[13747.522182] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 231b: must not assert on fully utilized OST request buffer ========================================================== 17:31:41 \(1734370301\)
[13747.868553] Lustre: DEBUG MARKER: == sanity test 231b: must not assert on fully utilized OST request buffer ========================================================== 17:31:41 (1734370301)
[13757.037914] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 232a: failed lock should not block umount ========================================================== 17:31:51 \(1734370311\)
[13757.360912] Lustre: DEBUG MARKER: == sanity test 232a: failed lock should not block umount ========================================================== 17:31:51 (1734370311)
[13757.884274] LustreError: lustre-OST0000-osc-ffff89618e2aa800: operation ldlm_enqueue to node 10.240.45.50@tcp failed: rc = -12
[13758.258389] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[13758.272608] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[13758.431601] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[13759.705990] Lustre: Unmounted lustre-client
[13759.719179] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[13759.732045] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[13759.809356] Lustre: Mounted lustre-client
[13778.334313] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 232b: failed data version lock should not block umount ========================================================== 17:32:12 \(1734370332\)
[13778.672028] Lustre: DEBUG MARKER: == sanity test 232b: failed data version lock should not block umount ========================================================== 17:32:12 (1734370332)
[13782.012041] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[13782.025192] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[13782.175308] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[13782.217646] Lustre: Unmounted lustre-client
[13782.229818] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[13782.243767] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[13782.301546] Lustre: Mounted lustre-client
[13787.538459] Lustre: lustre-OST0000-osc-ffff89619136c800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[13787.541409] Lustre: Skipped 2 previous similar messages
[13793.378426] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 233a: checking that OBF of the FS root succeeds ========================================================== 17:32:27 \(1734370347\)
[13793.725267] Lustre: DEBUG MARKER: == sanity test 233a: checking that OBF of the FS root succeeds ========================================================== 17:32:27 (1734370347)
[13795.574405] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 233b: checking that OBF of the FS .lustre succeeds ========================================================== 17:32:29 \(1734370349\)
[13795.946922] Lustre: DEBUG MARKER: == sanity test 233b: checking that OBF of the FS .lustre succeeds ========================================================== 17:32:29 (1734370349)
[13797.780759] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 234: xattr cache should not crash on ENOMEM ========================================================== 17:32:31 \(1734370351\)
[13798.129336] Lustre: DEBUG MARKER: == sanity test 234: xattr cache should not crash on ENOMEM ========================================================== 17:32:31 (1734370351)
[13798.271970] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[13798.271970] [[ -z "" ]] && param= ||
[13798.271970] param=$(grep <<< "$params");
[13798.271970] [[ -z $param ]] && param="$params";
[13798.271970] while read s; do echo client $s;
[13798.271970] done <<< "$param"
[13798.313373] Lustre: *** cfs_fail_loc=1405, val=0***
[13798.336922] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff89619136c800.xattr_cache=1
[13800.022945] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 235: LU-1715: flock deadlock detection does not work properly ========================================================== 17:32:34 \(1734370354\)
[13800.389757] Lustre: DEBUG MARKER: == sanity test 235: LU-1715: flock deadlock detection does not work properly ========================================================== 17:32:34 (1734370354)
[13804.251019] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 236: Layout swap on open unlinked file ==== 17:32:38 \(1734370358\)
[13804.576109] Lustre: DEBUG MARKER: == sanity test 236: Layout swap on open unlinked file ==== 17:32:38 (1734370358)
[13807.364945] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 238: Verify linkea consistency ============ 17:32:41 \(1734370361\)
[13807.724063] Lustre: DEBUG MARKER: == sanity test 238: Verify linkea consistency ============ 17:32:41 (1734370361)
[13809.635281] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 239A: osp_sync test ======================= 17:32:43 \(1734370363\)
[13809.984989] Lustre: DEBUG MARKER: == sanity test 239A: osp_sync test ======================= 17:32:43 (1734370363)
[13810.735205] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13815.533977] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13817.057779] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13819.818520] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13824.988216] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 239a: process invalid osp sync record correctly ========================================================== 17:32:59 \(1734370379\)
[13825.345375] Lustre: DEBUG MARKER: == sanity test 239a: process invalid osp sync record correctly ========================================================== 17:32:59 (1734370379)
[13840.870257] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 239b: process osp sync record with ENOMEM error correctly ========================================================== 17:33:14 \(1734370394\)
[13841.224129] Lustre: DEBUG MARKER: == sanity test 239b: process osp sync record with ENOMEM error correctly ========================================================== 17:33:14 (1734370394)
[13848.263376] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 240: race between ldlm enqueue and the connection RPC \(no ASSERT\) ========================================================== 17:33:22 \(1734370402\)
[13848.587304] Lustre: DEBUG MARKER: == sanity test 240: race between ldlm enqueue and the connection RPC (no ASSERT) ========================================================== 17:33:22 (1734370402)
[13848.995779] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_240 needs \>= 2 MDTs
[13849.337257] Lustre: DEBUG MARKER: SKIP: sanity test_240 needs >= 2 MDTs
[13849.769562] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 241a: bio vs dio ========================== 17:33:23 \(1734370403\)
[13850.117516] Lustre: DEBUG MARKER: == sanity test 241a: bio vs dio ========================== 17:33:23 (1734370403)
[13874.541737] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 241b: dio vs dio ========================== 17:33:48 \(1734370428\)
[13874.890471] Lustre: DEBUG MARKER: == sanity test 241b: dio vs dio ========================== 17:33:48 (1734370428)
[13881.589440] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 242: mdt_readpage failure should not cause directory unreadable ========================================================== 17:33:55 \(1734370435\)
[13881.970419] Lustre: DEBUG MARKER: == sanity test 242: mdt_readpage failure should not cause directory unreadable ========================================================== 17:33:55 (1734370435)
[13882.805870] LustreError: lustre-MDT0000-mdc-ffff89619136c800: operation mds_readpage to node 10.240.38.186@tcp failed: rc = -12
[13884.884931] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 243: various group lock tests ============= 17:33:58 \(1734370438\)
[13885.236919] Lustre: DEBUG MARKER: == sanity test 243: various group lock tests ============= 17:33:58 (1734370438)
[13886.544681] Lustre: 652060:0:(file.c:2984:ll_get_grouplock()) lustre: group lock already existed with gid 97486: rc = -22
[13886.546940] Lustre: 652060:0:(file.c:3058:ll_put_grouplock()) lustre: no group lock held: rc = -22
[13886.548651] Lustre: 652060:0:(file.c:2967:ll_get_grouplock()) lustre: group id for group lock must not be 0: rc = -22
[13886.551464] Lustre: 652060:0:(file.c:3067:ll_put_grouplock()) lustre: group lock 4294967286 not match current id 3543: rc = -22
[13922.194538] Lustre: 652060:0:(file.c:3058:ll_put_grouplock()) lustre: no group lock held: rc = -22
[13922.205408] Lustre: 652060:0:(file.c:2967:ll_get_grouplock()) lustre: group id for group lock must not be 0: rc = -22
[13924.104973] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 244a: sendfile with group lock tests ====== 17:34:38 \(1734370478\)
[13924.447600] Lustre: DEBUG MARKER: == sanity test 244a: sendfile with group lock tests ====== 17:34:38 (1734370478)
[13979.861107] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 244b: multi-threaded write with group lock ========================================================== 17:35:33 \(1734370533\)
[13980.217640] Lustre: DEBUG MARKER: == sanity test 244b: multi-threaded write with group lock ========================================================== 17:35:33 (1734370533)
[13983.643989] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 245a: check mdc connection flag\/data: multiple modify RPCs ========================================================== 17:35:37 \(1734370537\)
[13984.009219] Lustre: DEBUG MARKER: == sanity test 245a: check mdc connection flag/data: multiple modify RPCs ========================================================== 17:35:37 (1734370537)
[13985.838080] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 245b: check osp connection flag\/data: multiple modify RPCs ========================================================== 17:35:39 \(1734370539\)
[13986.179448] Lustre: DEBUG MARKER: == sanity test 245b: check osp connection flag/data: multiple modify RPCs ========================================================== 17:35:39 (1734370539)
[13986.649294] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_245b needs \>= 2 MDTs
[13986.974846] Lustre: DEBUG MARKER: SKIP: sanity test_245b needs >= 2 MDTs
[13987.421234] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247a: mount subdir as fileset ============= 17:35:41 \(1734370541\)
[13987.768991] Lustre: DEBUG MARKER: == sanity test 247a: mount subdir as fileset ============= 17:35:41 (1734370541)
[13988.782670] Autotest: Test running for 230 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[14004.020916] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247a.sanity
[14004.033585] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre_d247a.sanity
[14004.090441] Lustre: Mounted lustre-client
[14004.102413] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[14004.115699] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247a.sanity//d247a.sanity
[14004.132965] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247a.sanity' ' /proc/mounts ||
[14004.132965] umount /mnt/lustre_d247a.sanity
[14004.178358] Lustre: Unmounted lustre-client
[14004.185087] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d247a.sanity /mnt/lustre_d247a.sanity
[14004.335232] Lustre: DEBUG MARKER: grep -c /mnt/lustre_d247a.sanity' ' /proc/mounts
[14004.348009] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_d247a.sanity
[14004.505730] Lustre: DEBUG MARKER: umount /mnt/lustre_d247a.sanity 2>&1
[14006.348620] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247b: mount subdir that dose not exist ==== 17:36:00 \(1734370560\)
[14006.690688] Lustre: DEBUG MARKER: == sanity test 247b: mount subdir that dose not exist ==== 17:36:00 (1734370560)
[14006.846363] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247b.sanity
[14006.861440] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d247b.sanity /mnt/lustre_d247b.sanity
[14006.922373] LustreError: 654816:0:(llite_lib.c:710:client_common_fill_super()) cannot mds_connect: rc = -2
[14006.958124] Lustre: Unmounted lustre-client
[14006.958974] Lustre: Skipped 1 previous similar message
[14006.960555] LustreError: 654816:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -2
[14008.701464] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247c: running fid2path outside subdirectory root ========================================================== 17:36:02 \(1734370562\)
[14009.046834] Lustre: DEBUG MARKER: == sanity test 247c: running fid2path outside subdirectory root ========================================================== 17:36:02 (1734370562)
[14009.182940] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247c.sanity
[14009.196127] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre_d247c.sanity
[14009.257266] Lustre: Mounted lustre-client
[14009.258443] Lustre: Skipped 1 previous similar message
[14009.270072] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[14009.285453] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247c.sanity//d247c.sanity
[14009.304003] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247c.sanity' ' /proc/mounts ||
[14009.304003] umount /mnt/lustre_d247c.sanity
[14009.360191] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d247c.sanity /mnt/lustre_d247c.sanity
[14009.461883] Lustre: DEBUG MARKER: grep -c /mnt/lustre_d247c.sanity' ' /proc/mounts
[14009.474585] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_d247c.sanity
[14009.644460] Lustre: DEBUG MARKER: umount /mnt/lustre_d247c.sanity 2>&1
[14011.416088] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247d: running fid2path inside subdirectory root ========================================================== 17:36:05 \(1734370565\)
[14011.779401] Lustre: DEBUG MARKER: == sanity test 247d: running fid2path inside subdirectory root ========================================================== 17:36:05 (1734370565)
[14011.922269] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247d.sanity
[14011.934618] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre_d247d.sanity
[14012.003925] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[14012.019278] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247d.sanity//d247d.sanity
[14012.038717] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247d.sanity' ' /proc/mounts ||
[14012.038717] umount /mnt/lustre_d247d.sanity
[14012.084005] Lustre: Unmounted lustre-client
[14012.084872] Lustre: Skipped 2 previous similar messages
[14012.092498] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d247d.sanity /mnt/lustre_d247d.sanity
[14012.207908] Lustre: DEBUG MARKER: grep -c /mnt/lustre_d247d.sanity' ' /proc/mounts
[14012.221555] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_d247d.sanity
[14012.381516] Lustre: DEBUG MARKER: umount /mnt/lustre_d247d.sanity 2>&1
[14014.120340] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247e: mount .. as fileset ================= 17:36:08 \(1734370568\)
[14014.457579] Lustre: DEBUG MARKER: == sanity test 247e: mount .. as fileset ================= 17:36:08 (1734370568)
[14014.598795] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247e.sanity
[14014.611001] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre_d247e.sanity
[14014.664402] Lustre: Mounted lustre-client
[14014.665240] Lustre: Skipped 3 previous similar messages
[14014.675047] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[14014.690792] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247e.sanity//..
[14014.703982] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247e.sanity' ' /proc/mounts ||
[14014.703982] umount /mnt/lustre_d247e.sanity
[14014.755259] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/.. /mnt/lustre_d247e.sanity
[14014.809203] LustreError: lustre-MDT0000-mdc-ffff896190db4000: operation mds_get_root to node 10.240.38.186@tcp failed: rc = -22
[14014.811205] LustreError: 656378:0:(llite_lib.c:710:client_common_fill_super()) cannot mds_connect: rc = -22
[14014.842345] LustreError: 656378:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -22
[14016.501753] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247f: mount striped or remote directory as fileset ========================================================== 17:36:10 \(1734370570\)
[14016.844880] Lustre: DEBUG MARKER: == sanity test 247f: mount striped or remote directory as fileset ========================================================== 17:36:10 (1734370570)
[14017.259253] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_247f needs \>= 2 MDTs
[14017.590705] Lustre: DEBUG MARKER: SKIP: sanity test_247f needs >= 2 MDTs
[14018.060976] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247g: striped directory submount revalidate ROOT from cache ========================================================== 17:36:12 \(1734370572\)
[14018.372012] Lustre: DEBUG MARKER: == sanity test 247g: striped directory submount revalidate ROOT from cache ========================================================== 17:36:12 (1734370572)
[14018.805132] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_247g needs \> 1 MDTs
[14019.125767] Lustre: DEBUG MARKER: SKIP: sanity test_247g needs > 1 MDTs
[14019.563908] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247h: remote directory submount revalidate ROOT from cache ========================================================== 17:36:13 \(1734370573\)
[14019.908260] Lustre: DEBUG MARKER: == sanity test 247h: remote directory submount revalidate ROOT from cache ========================================================== 17:36:13 (1734370573)
[14020.330570] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_247h needs \> 1 MDTs
[14020.669489] Lustre: DEBUG MARKER: SKIP: sanity test_247h needs > 1 MDTs
[14021.096521] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 248a: fast read verification ============== 17:36:15 \(1734370575\)
[14021.475531] Lustre: DEBUG MARKER: == sanity test 248a: fast read verification ============== 17:36:15 (1734370575)
[14054.751927] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 248b: test short_io read and write for both small and large sizes ========================================================== 17:36:48 \(1734370608\)
[14055.120676] Lustre: DEBUG MARKER: == sanity test 248b: test short_io read and write for both small and large sizes ========================================================== 17:36:48 (1734370608)
[14290.128668] Autotest: Test running for 235 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[14310.800065] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 248c: verify whole file read behavior ===== 17:41:04 \(1734370864\)
[14311.157432] Lustre: DEBUG MARKER: == sanity test 248c: verify whole file read behavior ===== 17:41:04 (1734370864)
[14317.346022] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 249: Write above 2T file size ============= 17:41:11 \(1734370871\)
[14317.690401] Lustre: DEBUG MARKER: == sanity test 249: Write above 2T file size ============= 17:41:11 (1734370871)
[14319.620203] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 250: Write above 16T limit ================ 17:41:13 \(1734370873\)
[14319.951060] Lustre: DEBUG MARKER: == sanity test 250: Write above 16T limit ================ 17:41:13 (1734370873)
[14321.818499] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 251a: Handling short read and write correctly ========================================================== 17:41:15 \(1734370875\)
[14322.171371] Lustre: DEBUG MARKER: == sanity test 251a: Handling short read and write correctly ========================================================== 17:41:15 (1734370875)
[14322.744206] Lustre: *** cfs_fail_loc=1407, val=0***
[14324.490773] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 251b: short read restore offset correctly ========================================================== 17:41:18 \(1734370878\)
[14324.844355] Lustre: DEBUG MARKER: == sanity test 251b: short read restore offset correctly ========================================================== 17:41:18 (1734370878)
[14325.015634] LustreError: 660411:0:(file.c:2305:do_file_read_iter()) cfs_fail_timeout id 1431 sleeping for 5000ms
[14330.116832] LustreError: 660411:0:(file.c:2305:do_file_read_iter()) cfs_fail_timeout id 1431 awake
[14331.806066] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 252: check lr_reader tool ================= 17:41:25 \(1734370885\)
[14332.159934] Lustre: DEBUG MARKER: == sanity test 252: check lr_reader tool ================= 17:41:25 (1734370885)
[14335.837219] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 253: Check object allocation limit ======== 17:41:29 \(1734370889\)
[14336.191578] Lustre: DEBUG MARKER: == sanity test 253: Check object allocation limit ======== 17:41:29 (1734370889)
[14344.545938] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_253 2>/dev/null || echo foo
[14345.959727] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_253 |
[14345.959727] grep -e lustre-OST0000_UUID | sort -u | tr '\n' ' '
[14408.233396] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_253 2>/dev/null || echo foo
[14408.624875] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 254: Check changelog size ================= 17:42:42 \(1734370962\)
[14409.067113] Lustre: DEBUG MARKER: == sanity test 254: Check changelog size ================= 17:42:42 (1734370962)
[14414.463369] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 255a: check \'lfs ladvise -a willread\' ===== 17:42:48 \(1734370968\)
[14414.810382] Lustre: DEBUG MARKER: == sanity test 255a: check 'lfs ladvise -a willread' ===== 17:42:48 (1734370968)
[14591.147353] Autotest: Test running for 240 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[14893.046718] Autotest: Test running for 245 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15193.825247] Autotest: Test running for 250 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15408.089828] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 255b: check \'lfs ladvise -a dontneed\' ===== 17:59:22 \(1734371962\)
[15408.439642] Lustre: DEBUG MARKER: == sanity test 255b: check 'lfs ladvise -a dontneed' ===== 17:59:22 (1734371962)
[15426.199063] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 255c: suite of ladvise lockahead tests ==== 17:59:40 \(1734371980\)
[15426.557216] Lustre: DEBUG MARKER: == sanity test 255c: suite of ladvise lockahead tests ==== 17:59:40 (1734371980)
[15431.517885] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 256: Check llog delete for empty and not full state ========================================================== 17:59:45 \(1734371985\)
[15431.855079] Lustre: DEBUG MARKER: == sanity test 256: Check llog delete for empty and not full state ========================================================== 17:59:45 (1734371985)
[15442.467006] LustreError: lustre-MDT0000-mdc-ffff89619136c800: operation ldlm_enqueue to node 10.240.38.186@tcp failed: rc = -107
[15442.469088] Lustre: lustre-MDT0000-mdc-ffff89619136c800: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[15442.478203] Lustre: lustre-MDT0000-mdc-ffff89619136c800: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[15442.480020] Lustre: Skipped 3 previous similar messages
[15443.344674] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[15443.347930] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f560f9fd to 0x5d837553f56b5f54
[15446.732748] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 257: xattr locks are not lost ============= 18:00:00 \(1734372000\)
[15447.079401] Lustre: DEBUG MARKER: == sanity test 257: xattr locks are not lost ============= 18:00:00 (1734372000)
[15447.834300] LustreError: lustre-MDT0000-mdc-ffff89619136c800: operation ldlm_enqueue to node 10.240.38.186@tcp failed: rc = -14
[15453.586443] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[15453.589797] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f56b5f54 to 0x5d837553f56b618b
[15454.607744] Lustre: 429912:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734371992/real 1734371992] req@ffff8961d2ea2700 x1818616202338816/t0(0) o400->lustre-MDT0000-mdc-ffff89619136c800@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734372008 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[15454.612850] Lustre: 429912:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[15455.455519] Lustre: DEBUG MARKER: lctl get_param -n at_max
[15459.333133] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 258a: verify i_mutex security behavior when suid attributes is set ========================================================== 18:00:13 \(1734372013\)
[15459.673527] Lustre: DEBUG MARKER: == sanity test 258a: verify i_mutex security behavior when suid attributes is set ========================================================== 18:00:13 (1734372013)
[15461.497493] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 258b: verify i_mutex security behavior ==== 18:00:15 \(1734372015\)
[15461.842134] Lustre: DEBUG MARKER: == sanity test 258b: verify i_mutex security behavior ==== 18:00:15 (1734372015)
[15463.719123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 259: crash at delayed truncate ============ 18:00:17 \(1734372017\)
[15464.052790] Lustre: DEBUG MARKER: == sanity test 259: crash at delayed truncate ============ 18:00:17 (1734372017)
[15495.495191] Autotest: Test running for 255 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15496.475217] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 260: Check mdc_close fail ================= 18:00:50 \(1734372050\)
[15496.818571] Lustre: DEBUG MARKER: == sanity test 260: Check mdc_close fail ================= 18:00:50 (1734372050)
[15496.977835] Lustre: *** cfs_fail_loc=806, val=0***
[15496.978740] Lustre: 671198:0:(mdc_request.c:922:mdc_close()) lustre-MDT0000-mdc-ffff89619136c800: close of FID [0x2000061c3:0x1412:0x0] failed, file reference will be dropped when this client unmounts or is evicted
[15496.981742] LustreError: 671198:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff89619136c800: inode [0x2000061c3:0x1412:0x0] mdc close failed: rc = -12
[15498.670845] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270a: DoM: basic functionality tests ====== 18:00:52 \(1734372052\)
[15499.014474] Lustre: DEBUG MARKER: == sanity test 270a: DoM: basic functionality tests ====== 18:00:52 (1734372052)
[15506.812462] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270b: DoM: maximum size overflow checks for DoM-only file ========================================================== 18:01:00 \(1734372060\)
[15507.167169] Lustre: DEBUG MARKER: == sanity test 270b: DoM: maximum size overflow checks for DoM-only file ========================================================== 18:01:00 (1734372060)
[15509.075392] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270c: DoM: DoM EA inheritance tests ======= 18:01:03 \(1734372063\)
[15509.446023] Lustre: DEBUG MARKER: == sanity test 270c: DoM: DoM EA inheritance tests ======= 18:01:03 (1734372063)
[15511.341928] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270d: DoM: change striping from DoM to RAID0 ========================================================== 18:01:05 \(1734372065\)
[15511.685328] Lustre: DEBUG MARKER: == sanity test 270d: DoM: change striping from DoM to RAID0 ========================================================== 18:01:05 (1734372065)
[15513.659944] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270e: DoM: lfs find with DoM files test === 18:01:07 \(1734372067\)
[15514.021969] Lustre: DEBUG MARKER: == sanity test 270e: DoM: lfs find with DoM files test === 18:01:07 (1734372067)
[15516.123523] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270f: DoM: maximum DoM stripe size checks ========================================================== 18:01:10 \(1734372070\)
[15516.451947] Lustre: DEBUG MARKER: == sanity test 270f: DoM: maximum DoM stripe size checks ========================================================== 18:01:10 (1734372070)
[15522.737506] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270g: DoM: default DoM stripe size depends on free space ========================================================== 18:01:16 \(1734372076\)
[15523.081949] Lustre: DEBUG MARKER: == sanity test 270g: DoM: default DoM stripe size depends on free space ========================================================== 18:01:16 (1734372076)
[15532.706091] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270h: DoM: DoM stripe removal when disabled on server ========================================================== 18:01:26 \(1734372086\)
[15533.059705] Lustre: DEBUG MARKER: == sanity test 270h: DoM: DoM stripe removal when disabled on server ========================================================== 18:01:26 (1734372086)
[15536.202034] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270i: DoM: setting invalid DoM striping should fail ========================================================== 18:01:30 \(1734372090\)
[15536.558370] Lustre: DEBUG MARKER: == sanity test 270i: DoM: setting invalid DoM striping should fail ========================================================== 18:01:30 (1734372090)
[15538.448001] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270j: DoM migration: DOM file to the OST-striped file \(plain\) ========================================================== 18:01:32 \(1734372092\)
[15538.810243] Lustre: DEBUG MARKER: == sanity test 270j: DoM migration: DOM file to the OST-striped file (plain) ========================================================== 18:01:32 (1734372092)
[15540.891086] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271a: DoM: data is cached for read after write ========================================================== 18:01:34 \(1734372094\)
[15541.266240] Lustre: DEBUG MARKER: == sanity test 271a: DoM: data is cached for read after write ========================================================== 18:01:34 (1734372094)
[15543.163409] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271b: DoM: no glimpse RPC for stat \(DoM only file\) ========================================================== 18:01:37 \(1734372097\)
[15543.487408] Lustre: DEBUG MARKER: == sanity test 271b: DoM: no glimpse RPC for stat (DoM only file) ========================================================== 18:01:37 (1734372097)
[15545.412507] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271ba: DoM: no glimpse RPC for stat \(combined file\) ========================================================== 18:01:39 \(1734372099\)
[15545.748437] Lustre: DEBUG MARKER: == sanity test 271ba: DoM: no glimpse RPC for stat (combined file) ========================================================== 18:01:39 (1734372099)
[15547.702801] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271c: DoM: IO lock at open saves enqueue RPCs ========================================================== 18:01:41 \(1734372101\)
[15548.047010] Lustre: DEBUG MARKER: == sanity test 271c: DoM: IO lock at open saves enqueue RPCs ========================================================== 18:01:41 (1734372101)
[15549.148747] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[15550.655065] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[15558.330029] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[15559.738800] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[15561.590580] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[15563.067931] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[15568.205911] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[15569.852377] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[15572.497948] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271d: DoM: read on open \(1K file in reply buffer\) ========================================================== 18:02:06 \(1734372126\)
[15572.838178] Lustre: DEBUG MARKER: == sanity test 271d: DoM: read on open (1K file in reply buffer) ========================================================== 18:02:06 (1734372126)
[15574.882052] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271f: DoM: read on open \(200K file and read tail\) ========================================================== 18:02:08 \(1734372128\)
[15575.234828] Lustre: DEBUG MARKER: == sanity test 271f: DoM: read on open (200K file and read tail) ========================================================== 18:02:08 (1734372128)
[15577.258678] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271g: Discard DoM data vs client flush race ========================================================== 18:02:11 \(1734372131\)
[15577.601304] Lustre: DEBUG MARKER: == sanity test 271g: Discard DoM data vs client flush race ========================================================== 18:02:11 (1734372131)
[15578.760144] Lustre: *** cfs_fail_loc=314, val=0***
[15580.462602] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272a: DoM migration: new layout with the same DOM component ========================================================== 18:02:14 \(1734372134\)
[15580.811300] Lustre: DEBUG MARKER: == sanity test 272a: DoM migration: new layout with the same DOM component ========================================================== 18:02:14 (1734372134)
[15585.470099] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272b: DoM migration: DOM file to the OST-striped file \(plain\) ========================================================== 18:02:19 \(1734372139\)
[15585.823210] Lustre: DEBUG MARKER: == sanity test 272b: DoM migration: DOM file to the OST-striped file (plain) ========================================================== 18:02:19 (1734372139)
[15590.540785] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272c: DoM migration: DOM file to the OST-striped file \(composite\) ========================================================== 18:02:24 \(1734372144\)
[15590.900761] Lustre: DEBUG MARKER: == sanity test 272c: DoM migration: DOM file to the OST-striped file (composite) ========================================================== 18:02:24 (1734372144)
[15595.436052] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272d: DoM mirroring: OST-striped mirror to DOM file ========================================================== 18:02:29 \(1734372149\)
[15595.789639] Lustre: DEBUG MARKER: == sanity test 272d: DoM mirroring: OST-striped mirror to DOM file ========================================================== 18:02:29 (1734372149)
[15597.682236] LustreError: 682246:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff89619136c800: inode [0x2000061c3:0x1c4a:0x0] mdc close failed: rc = -22
[15599.910537] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272e: DoM mirroring: DOM mirror to the OST-striped file ========================================================== 18:02:33 \(1734372153\)
[15600.242648] Lustre: DEBUG MARKER: == sanity test 272e: DoM mirroring: DOM mirror to the OST-striped file ========================================================== 18:02:33 (1734372153)
[15601.218438] LustreError: 682722:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff89619136c800: inode [0x2000061c3:0x1c4e:0x0] mdc close failed: rc = -22
[15603.031437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272f: DoM migration: OST-striped file to DOM file ========================================================== 18:02:37 \(1734372157\)
[15603.366332] Lustre: DEBUG MARKER: == sanity test 272f: DoM migration: OST-striped file to DOM file ========================================================== 18:02:37 (1734372157)
[15604.600823] LustreError: 683186:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff89619136c800: inode [0x2000061c3:0x1c53:0x0] mdc close failed: rc = -22
[15606.423488] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 273a: DoM: layout swapping should fail with DOM ========================================================== 18:02:40 \(1734372160\)
[15606.793745] Lustre: DEBUG MARKER: == sanity test 273a: DoM: layout swapping should fail with DOM ========================================================== 18:02:40 (1734372160)
[15608.784310] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 273b: DoM: race writeback and object destroy ========================================================== 18:02:42 \(1734372162\)
[15609.117479] Lustre: DEBUG MARKER: == sanity test 273b: DoM: race writeback and object destroy ========================================================== 18:02:42 (1734372162)
[15613.834891] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 273c: race writeback and object destroy === 18:02:47 \(1734372167\)
[15614.184371] Lustre: DEBUG MARKER: == sanity test 273c: race writeback and object destroy === 18:02:47 (1734372167)
[15616.851467] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 275: Read on a canceled duplicate lock ==== 18:02:50 \(1734372170\)
[15617.209698] Lustre: DEBUG MARKER: == sanity test 275: Read on a canceled duplicate lock ==== 18:02:50 (1734372170)
[15617.618579] LustreError: 653052:0:(ldlm_lockd.c:2927:ldlm_bl_thread_blwi()) cfs_fail_timeout id 31f sleeping for 4000ms
[15619.392169] LustreError: 653052:0:(ldlm_lockd.c:2927:ldlm_bl_thread_blwi()) cfs_fail_timeout interrupted
[15620.760338] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 276: Race between mount and obd_statfs ==== 18:02:54 \(1734372174\)
[15621.109512] Lustre: DEBUG MARKER: == sanity test 276: Race between mount and obd_statfs ==== 18:02:54 (1734372174)
[15622.547208] Lustre: lustre-OST0000-osc-ffff89619136c800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[15622.550063] Lustre: Skipped 2 previous similar messages
[15638.788986] Lustre: lustre-OST0000-osc-ffff89619136c800: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[15638.790860] Lustre: Skipped 3 previous similar messages
[15707.717689] Lustre: lustre-OST0000-osc-ffff89619136c800: Connection restored to 10.240.45.50@tcp (at 10.240.45.50@tcp)
[15707.724450] Lustre: Skipped 6 previous similar messages
[15753.621054] Lustre: lustre-OST0000-osc-ffff89619136c800: Connection to lustre-OST0000 (at 10.240.45.50@tcp) was lost; in progress operations using this service will wait for recovery to complete
[15753.628314] Lustre: Skipped 12 previous similar messages
[15796.376541] Autotest: Test running for 260 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15821.842267] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 277: Direct IO shall drop page cache ====== 18:06:15 \(1734372375\)
[15822.195827] Lustre: DEBUG MARKER: == sanity test 277: Direct IO shall drop page cache ====== 18:06:15 (1734372375)
[15824.318351] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 278: Race starting MDS between MDTs stop\/start ========================================================== 18:06:18 \(1734372378\)
[15824.661686] Lustre: DEBUG MARKER: == sanity test 278: Race starting MDS between MDTs stop/start ========================================================== 18:06:18 (1734372378)
[15825.069235] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_278 needs \>= 2 MDTs
[15825.418040] Lustre: DEBUG MARKER: SKIP: sanity test_278 needs >= 2 MDTs
[15825.882765] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 280: Race between MGS umount and client llog processing ========================================================== 18:06:19 \(1734372379\)
[15826.209715] Lustre: DEBUG MARKER: == sanity test 280: Race between MGS umount and client llog processing ========================================================== 18:06:19 (1734372379)
[15826.374111] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[15826.387157] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[15826.562492] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[15826.635187] Lustre: Unmounted lustre-client
[15826.636021] Lustre: Skipped 3 previous similar messages
[15827.012841] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[15827.025546] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[15837.329464] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[15837.332277] LustreError: MGC10.240.38.186@tcp: Confguration from log lustre-client failed from MGS -5. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info
[15837.335914] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f56eca70 to 0x5d837553f56ecc92
[15837.341664] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[15837.343266] Lustre: Skipped 11 previous similar messages
[15837.347498] Lustre: Unmounted lustre-client
[15837.350057] LustreError: 692217:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -5
[15865.872841] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[15865.884973] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[15881.274186] Lustre: Mounted lustre-client
[15883.338109] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300a: basic striped dir sanity test ======= 18:07:17 \(1734372437\)
[15883.746709] Lustre: DEBUG MARKER: == sanity test 300a: basic striped dir sanity test ======= 18:07:17 (1734372437)
[15884.202931] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300a needs \>= 2 MDTs
[15884.615644] Lustre: DEBUG MARKER: SKIP: sanity test_300a needs >= 2 MDTs
[15885.116608] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300b: check ctime\/mtime for striped dir === 18:07:19 \(1734372439\)
[15885.513383] Lustre: DEBUG MARKER: == sanity test 300b: check ctime/mtime for striped dir === 18:07:19 (1734372439)
[15886.021114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300b needs \>= 2 MDTs
[15886.461633] Lustre: DEBUG MARKER: SKIP: sanity test_300b needs >= 2 MDTs
[15886.959287] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300c: chown && check ls under striped directory ========================================================== 18:07:21 \(1734372441\)
[15887.368515] Lustre: DEBUG MARKER: == sanity test 300c: chown
[15887.874612] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300c needs \>= 2 MDTs
[15888.247079] Lustre: DEBUG MARKER: SKIP: sanity test_300c needs >= 2 MDTs
[15888.706224] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300d: check default stripe under striped directory ========================================================== 18:07:22 \(1734372442\)
[15889.134500] Lustre: DEBUG MARKER: == sanity test 300d: check default stripe under striped directory ========================================================== 18:07:22 (1734372442)
[15889.606068] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300d needs \>= 2 MDTs
[15889.946625] Lustre: DEBUG MARKER: SKIP: sanity test_300d needs >= 2 MDTs
[15890.427886] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300e: check rename under striped directory ========================================================== 18:07:24 \(1734372444\)
[15890.828338] Lustre: DEBUG MARKER: == sanity test 300e: check rename under striped directory ========================================================== 18:07:24 (1734372444)
[15891.328786] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300e needs \>= 2 MDTs
[15891.673617] Lustre: DEBUG MARKER: SKIP: sanity test_300e needs >= 2 MDTs
[15892.169814] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300f: check rename cross striped directory ========================================================== 18:07:26 \(1734372446\)
[15892.534350] Lustre: DEBUG MARKER: == sanity test 300f: check rename cross striped directory ========================================================== 18:07:26 (1734372446)
[15892.971293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300f needs \>= 2 MDTs
[15893.363328] Lustre: DEBUG MARKER: SKIP: sanity test_300f needs >= 2 MDTs
[15893.842043] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300g: check default striped directory for normal directory ========================================================== 18:07:27 \(1734372447\)
[15894.218338] Lustre: DEBUG MARKER: == sanity test 300g: check default striped directory for normal directory ========================================================== 18:07:27 (1734372447)
[15894.652231] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300g needs \>= 2 MDTs
[15895.049767] Lustre: DEBUG MARKER: SKIP: sanity test_300g needs >= 2 MDTs
[15895.498909] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300h: check default striped directory for striped directory ========================================================== 18:07:29 \(1734372449\)
[15895.903595] Lustre: DEBUG MARKER: == sanity test 300h: check default striped directory for striped directory ========================================================== 18:07:29 (1734372449)
[15896.304572] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300h needs \>= 2 MDTs
[15896.671513] Lustre: DEBUG MARKER: SKIP: sanity test_300h needs >= 2 MDTs
[15897.168713] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300i: client handle unknown hash type striped directory ========================================================== 18:07:31 \(1734372451\)
[15897.510872] Lustre: DEBUG MARKER: == sanity test 300i: client handle unknown hash type striped directory ========================================================== 18:07:31 (1734372451)
[15897.958096] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300i needs \>= 2 MDTs
[15898.305788] Lustre: DEBUG MARKER: SKIP: sanity test_300i needs >= 2 MDTs
[15898.794531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300j: test large update record ============ 18:07:32 \(1734372452\)
[15899.154149] Lustre: DEBUG MARKER: == sanity test 300j: test large update record ============ 18:07:32 (1734372452)
[15899.586960] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300j needs \>= 2 MDTs
[15899.966066] Lustre: DEBUG MARKER: SKIP: sanity test_300j needs >= 2 MDTs
[15900.443674] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300k: test large striped directory ======== 18:07:34 \(1734372454\)
[15900.839621] Lustre: DEBUG MARKER: == sanity test 300k: test large striped directory ======== 18:07:34 (1734372454)
[15901.292182] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300k needs \>= 2 MDTs
[15901.691879] Lustre: DEBUG MARKER: SKIP: sanity test_300k needs >= 2 MDTs
[15902.196799] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300l: non-root user to create dir under striped dir with stale layout ========================================================== 18:07:36 \(1734372456\)
[15902.616438] Lustre: DEBUG MARKER: == sanity test 300l: non-root user to create dir under striped dir with stale layout ========================================================== 18:07:36 (1734372456)
[15903.047421] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300l needs \>= 2 MDTs
[15903.440244] Lustre: DEBUG MARKER: SKIP: sanity test_300l needs >= 2 MDTs
[15903.955118] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300m: setstriped directory on single MDT FS ========================================================== 18:07:38 \(1734372458\)
[15904.371441] Lustre: DEBUG MARKER: == sanity test 300m: setstriped directory on single MDT FS ========================================================== 18:07:38 (1734372458)
[15906.484654] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300n: non-root user to create dir under striped dir with default EA ========================================================== 18:07:40 \(1734372460\)
[15906.874197] Lustre: DEBUG MARKER: == sanity test 300n: non-root user to create dir under striped dir with default EA ========================================================== 18:07:40 (1734372460)
[15907.330136] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300n needs \>= 2 MDTs
[15907.691513] Lustre: DEBUG MARKER: SKIP: sanity test_300n needs >= 2 MDTs
[15908.154242] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300o skipping SLOW test 300o
[15908.562906] Lustre: DEBUG MARKER: SKIP: sanity test_300o skipping SLOW test 300o
[15908.995299] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300p: create striped directory without space ========================================================== 18:07:43 \(1734372463\)
[15909.364093] Lustre: DEBUG MARKER: == sanity test 300p: create striped directory without space ========================================================== 18:07:43 (1734372463)
[15909.786684] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300p needs \>= 2 MDTs
[15910.171365] Lustre: DEBUG MARKER: SKIP: sanity test_300p needs >= 2 MDTs
[15910.672847] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300q: create remote directory under orphan directory ========================================================== 18:07:44 \(1734372464\)
[15911.023059] Lustre: DEBUG MARKER: == sanity test 300q: create remote directory under orphan directory ========================================================== 18:07:44 (1734372464)
[15911.441564] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300q needs \>= 2 MDTs
[15911.803683] Lustre: DEBUG MARKER: SKIP: sanity test_300q needs >= 2 MDTs
[15912.263279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300r: test -1 striped directory =========== 18:07:46 \(1734372466\)
[15912.644807] Lustre: DEBUG MARKER: == sanity test 300r: test -1 striped directory =========== 18:07:46 (1734372466)
[15913.072622] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300r needs \>= 2 MDTs
[15913.453441] Lustre: DEBUG MARKER: SKIP: sanity test_300r needs >= 2 MDTs
[15913.915983] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300s: test lfs mkdir -c without -i ======== 18:07:47 \(1734372467\)
[15914.278545] Lustre: DEBUG MARKER: == sanity test 300s: test lfs mkdir -c without -i ======== 18:07:47 (1734372467)
[15914.721028] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300s needs \>= 2 MDTs
[15915.095571] Lustre: DEBUG MARKER: SKIP: sanity test_300s needs >= 2 MDTs
[15915.571069] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300t: test max_mdt_stripecount ============ 18:07:49 \(1734372469\)
[15916.108616] Lustre: DEBUG MARKER: == sanity test 300t: test max_mdt_stripecount ============ 18:07:49 (1734372469)
[15916.932797] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300t needs at least 2 MDTs
[15917.847431] Lustre: DEBUG MARKER: SKIP: sanity test_300t needs at least 2 MDTs
[15918.966722] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ua: basic overstriped dir sanity test == 18:07:53 \(1734372473\)
[15919.340984] Lustre: DEBUG MARKER: == sanity test 300ua: basic overstriped dir sanity test == 18:07:53 (1734372473)
[15919.757129] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300ua needs \>= 2 MDTs
[15920.123128] Lustre: DEBUG MARKER: SKIP: sanity test_300ua needs >= 2 MDTs
[15920.571293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ub: test MDT overstriping interface & limits ========================================================== 18:07:54 \(1734372474\)
[15920.915278] Lustre: DEBUG MARKER: == sanity test 300ub: test MDT overstriping interface
[15921.325756] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300ub needs \>= 2 MDTs
[15921.679363] Lustre: DEBUG MARKER: SKIP: sanity test_300ub needs >= 2 MDTs
[15922.158082] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300uc: test MDT overstriping as default & inheritance ========================================================== 18:07:56 \(1734372476\)
[15922.512703] Lustre: DEBUG MARKER: == sanity test 300uc: test MDT overstriping as default
[15922.908618] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300uc needs \>= 2 MDTs
[15923.260308] Lustre: DEBUG MARKER: SKIP: sanity test_300uc needs >= 2 MDTs
[15923.711124] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ud: dir split ========================== 18:07:57 \(1734372477\)
[15924.068690] Lustre: DEBUG MARKER: == sanity test 300ud: dir split ========================== 18:07:57 (1734372477)
[15924.491643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300ud needs \>= 2 MDTs
[15924.853041] Lustre: DEBUG MARKER: SKIP: sanity test_300ud needs >= 2 MDTs
[15925.311482] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ue: dir merge ========================== 18:07:59 \(1734372479\)
[15925.674330] Lustre: DEBUG MARKER: == sanity test 300ue: dir merge ========================== 18:07:59 (1734372479)
[15926.088361] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300ue needs \>= 2 MDTs
[15926.460155] Lustre: DEBUG MARKER: SKIP: sanity test_300ue needs >= 2 MDTs
[15926.933674] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300uf: migrate with too many local locks == 18:08:01 \(1734372481\)
[15927.291428] Lustre: DEBUG MARKER: == sanity test 300uf: migrate with too many local locks == 18:08:01 (1734372481)
[15927.729394] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300uf needs \>= 2 MDTs
[15928.108123] Lustre: DEBUG MARKER: SKIP: sanity test_300uf needs >= 2 MDTs
[15928.575923] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ug: migrate overstriped dirs =========== 18:08:02 \(1734372482\)
[15928.936947] Lustre: DEBUG MARKER: == sanity test 300ug: migrate overstriped dirs =========== 18:08:02 (1734372482)
[15929.373108] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300ug needs \>= 2 MDTs
[15929.762987] Lustre: DEBUG MARKER: SKIP: sanity test_300ug needs >= 2 MDTs
[15930.241698] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300uh: overstripe tunable max_stripes_per_mdt ========================================================== 18:08:04 \(1734372484\)
[15930.624899] Lustre: DEBUG MARKER: == sanity test 300uh: overstripe tunable max_stripes_per_mdt ========================================================== 18:08:04 (1734372484)
[15931.052675] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_300uh needs \>= 2 MDTs
[15931.408029] Lustre: DEBUG MARKER: SKIP: sanity test_300uh needs >= 2 MDTs
[15931.878812] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ui: overstripe is not supported on one MDT system ========================================================== 18:08:05 \(1734372485\)
[15932.238571] Lustre: DEBUG MARKER: == sanity test 300ui: overstripe is not supported on one MDT system ========================================================== 18:08:05 (1734372485)
[15934.530182] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 310a: open unlink remote file ============= 18:08:08 \(1734372488\)
[15934.901093] Lustre: DEBUG MARKER: == sanity test 310a: open unlink remote file ============= 18:08:08 (1734372488)
[15935.316869] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_310a needs \>= 4 MDTs
[15935.675280] Lustre: DEBUG MARKER: SKIP: sanity test_310a needs >= 4 MDTs
[15936.139327] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 310b: unlink remote file with multiple links while open ========================================================== 18:08:10 \(1734372490\)
[15936.501906] Lustre: DEBUG MARKER: == sanity test 310b: unlink remote file with multiple links while open ========================================================== 18:08:10 (1734372490)
[15936.939570] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_310b needs \>= 4 MDTs
[15937.294092] Lustre: DEBUG MARKER: SKIP: sanity test_310b needs >= 4 MDTs
[15937.781913] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 310c: open-unlink remote file with multiple links ========================================================== 18:08:11 \(1734372491\)
[15938.141508] Lustre: DEBUG MARKER: == sanity test 310c: open-unlink remote file with multiple links ========================================================== 18:08:11 (1734372491)
[15938.580299] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_310c needs \>= 4 MDTs
[15938.946767] Lustre: DEBUG MARKER: SKIP: sanity test_310c needs >= 4 MDTs
[15939.419341] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 311: disable OSP precreate, and unlink should destroy objs ========================================================== 18:08:13 \(1734372493\)
[15939.804739] Lustre: DEBUG MARKER: == sanity test 311: disable OSP precreate, and unlink should destroy objs ========================================================== 18:08:13 (1734372493)
[15940.657146] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[15942.239811] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[15944.893456] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[15945.992511] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[15962.911057] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 312: make sure ZFS adjusts its block size by write pattern ========================================================== 18:08:36 \(1734372516\)
[15963.293341] Lustre: DEBUG MARKER: == sanity test 312: make sure ZFS adjusts its block size by write pattern ========================================================== 18:08:36 (1734372516)
[15963.785144] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_312 the test only applies to zfs
[15964.173878] Lustre: DEBUG MARKER: SKIP: sanity test_312 the test only applies to zfs
[15964.671887] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 313: io should fail after last_rcvd update fail ========================================================== 18:08:38 \(1734372518\)
[15965.056187] Lustre: DEBUG MARKER: == sanity test 313: io should fail after last_rcvd update fail ========================================================== 18:08:38 (1734372518)
[15967.746056] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 314: OSP shouldn\'t fail after last_rcvd update failure ========================================================== 18:08:41 \(1734372521\)
[15968.105439] Lustre: DEBUG MARKER: == sanity test 314: OSP shouldn't fail after last_rcvd update failure ========================================================== 18:08:41 (1734372521)
[15982.590046] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 315: read should be accounted ============= 18:08:56 \(1734372536\)
[15982.972533] Lustre: DEBUG MARKER: == sanity test 315: read should be accounted ============= 18:08:56 (1734372536)
[15989.057489] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 316: lfs migrate of file with large_xattr enabled ========================================================== 18:09:03 \(1734372543\)
[15989.452474] Lustre: DEBUG MARKER: == sanity test 316: lfs migrate of file with large_xattr enabled ========================================================== 18:09:03 (1734372543)
[15989.880428] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_316 needs \>= 2 MDTs
[15990.276767] Lustre: DEBUG MARKER: SKIP: sanity test_316 needs >= 2 MDTs
[15990.752951] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 317: Verify blocks get correctly update after truncate ========================================================== 18:09:04 \(1734372544\)
[15991.164330] Lustre: DEBUG MARKER: == sanity test 317: Verify blocks get correctly update after truncate ========================================================== 18:09:04 (1734372544)
[15994.140008] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 318: Verify async readahead tunables ====== 18:09:08 \(1734372548\)
[15994.526490] Lustre: DEBUG MARKER: == sanity test 318: Verify async readahead tunables ====== 18:09:08 (1734372548)
[15994.786819] LustreError: 705748:0:(lproc_llite.c:1760:read_ahead_async_file_threshold_mb_store()) lustre: can't set read_ahead_async_file_threshold_mb=65 > max_read_readahead_per_file_mb=64
[15996.657050] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 319: lost lease lock on migrate error ===== 18:09:10 \(1734372550\)
[15997.107490] Lustre: DEBUG MARKER: == sanity test 319: lost lease lock on migrate error ===== 18:09:10 (1734372550)
[15997.628523] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_319 needs \>= 2 MDTs
[15998.057633] Lustre: DEBUG MARKER: SKIP: sanity test_319 needs >= 2 MDTs
[15998.634172] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 350: force NID mismatch path to be exercised ========================================================== 18:09:12 \(1734372552\)
[15999.021055] Lustre: DEBUG MARKER: == sanity test 350: force NID mismatch path to be exercised ========================================================== 18:09:12 (1734372552)
[16033.958384] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 360: ldiskfs unlink in a separate thread == 18:09:48 \(1734372588\)
[16034.340522] Lustre: DEBUG MARKER: == sanity test 360: ldiskfs unlink in a separate thread == 18:09:48 (1734372588)
[16051.217898] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398a: direct IO should cancel lock otherwise lockless ========================================================== 18:10:05 \(1734372605\)
[16051.591238] Lustre: DEBUG MARKER: == sanity test 398a: direct IO should cancel lock otherwise lockless ========================================================== 18:10:05 (1734372605)
[16053.946165] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398b: DIO and buffer IO race ============== 18:10:08 \(1734372608\)
[16054.306113] Lustre: DEBUG MARKER: == sanity test 398b: DIO and buffer IO race ============== 18:10:08 (1734372608)
[16097.137114] Autotest: Test running for 265 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[16398.134669] Autotest: Test running for 270 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[16500.986819] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST*-osc-*/state
[16502.820238] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398c: run fio to test AIO ================= 18:17:36 \(1734373056\)
[16503.168973] Lustre: DEBUG MARKER: == sanity test 398c: run fio to test AIO ================= 18:17:36 (1734373056)
[16699.055318] Autotest: Test running for 275 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[17000.134244] Autotest: Test running for 280 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[17108.422275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398d: run aiocp to verify block size \> stripe size ========================================================== 18:27:42 \(1734373662\)
[17108.775666] Lustre: DEBUG MARKER: == sanity test 398d: run aiocp to verify block size > stripe size ========================================================== 18:27:42 (1734373662)
[17135.659498] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398e: O_Direct open cleared by fcntl doesn\'t cause hang ========================================================== 18:28:09 \(1734373689\)
[17136.010969] Lustre: DEBUG MARKER: == sanity test 398e: O_Direct open cleared by fcntl doesn't cause hang ========================================================== 18:28:09 (1734373689)
[17140.642329] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398f: verify aio handles ll_direct_rw_pages errors correctly ========================================================== 18:28:14 \(1734373694\)
[17140.997700] Lustre: DEBUG MARKER: == sanity test 398f: verify aio handles ll_direct_rw_pages errors correctly ========================================================== 18:28:14 (1734373694)
[17156.163950] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398g: verify parallel dio async RPC submission ========================================================== 18:28:30 \(1734373710\)
[17156.515314] Lustre: DEBUG MARKER: == sanity test 398g: verify parallel dio async RPC submission ========================================================== 18:28:30 (1734373710)
[17207.986271] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398h: verify correctness of read & write with i\/o size \>\> stripe size ========================================================== 18:29:22 \(1734373762\)
[17208.334485] Lustre: DEBUG MARKER: == sanity test 398h: verify correctness of read
<ConMan> Console [trevis-106vm14] log opened at 2024-12-16 18:32:11 UTC.
<ConMan> Console [trevis-106vm14] connected to <trevis-106:6013>.
[17432.456894] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398k: test enospc on first stripe ========= 18:33:06 \(1734373986\)
[17432.838065] Lustre: DEBUG MARKER: == sanity test 398k: test enospc on first stripe ========= 18:33:06 (1734373986)
[17452.291294] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_398k 49717352 \> 2100000 skipping out-of-space test on OST0
[17452.754860] Lustre: DEBUG MARKER: SKIP: sanity test_398k 49717352 > 2100000 skipping out-of-space test on OST0
[17453.186364] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398l: test enospc on intermediate stripe\/RPC ========================================================== 18:33:27 \(1734374007\)
[17453.534475] Lustre: DEBUG MARKER: == sanity test 398l: test enospc on intermediate stripe/RPC ========================================================== 18:33:27 (1734374007)
[17467.043854] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_398l 49697632 \> 2100000 skipping out-of-space test on OST0
[17467.459618] Lustre: DEBUG MARKER: SKIP: sanity test_398l 49697632 > 2100000 skipping out-of-space test on OST0
[17485.381608] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398m: test RPC failures with parallel dio ========================================================== 18:33:59 \(1734374039\)
[17485.782449] Lustre: DEBUG MARKER: == sanity test 398m: test RPC failures with parallel dio ========================================================== 18:33:59 (1734374039)
[17486.337540] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17486.339558] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961834dd380 x1818616221988224/t0(0) o4->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374095 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'dd.0' uid:0 gid:0
[17486.933992] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17486.935890] LustreError: Skipped 3 previous similar messages
[17486.936920] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961834dcd00 x1818616221988992/t0(0) o4->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374096 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_00.0' uid:0 gid:0
[17486.941393] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 3 previous similar messages
[17488.951467] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17488.953439] LustreError: Skipped 3 previous similar messages
[17488.954477] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961834de3c0 x1818616221990912/t0(0) o4->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374098 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_00.0' uid:0 gid:0
[17488.958979] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 3 previous similar messages
[17492.117832] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17492.117928] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961834dcd00 x1818616221991424/t0(0) o4->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374101 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_00.0' uid:0 gid:0
[17492.119707] LustreError: Skipped 4 previous similar messages
[17492.125279] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 6 previous similar messages
[17496.151615] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17496.153564] LustreError: Skipped 2 previous similar messages
[17496.154608] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961834dd380 x1818616221993344/t0(0) o4->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374105 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_00.0' uid:0 gid:0
[17507.447764] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17507.449663] LustreError: Skipped 7 previous similar messages
[17507.450704] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961917863c0 x1818616221995264/t0(0) o4->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374116 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_01.0' uid:0 gid:0
[17507.455090] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 7 previous similar messages
[17531.865167] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17531.867089] LustreError: Skipped 11 previous similar messages
[17531.868167] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961917870c0 x1818616221999744/t0(0) o4->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374141 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_01.0' uid:0 gid:0
[17531.872627] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 11 previous similar messages
[17542.037964] LustreError: 429911:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff896228103000: too many resent retries for object: 9663685440:8792: rc = -5
[17601.801224] Autotest: Test running for 290 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[17603.967697] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_read to node 10.240.45.50@tcp failed: rc = -5
[17603.969601] LustreError: Skipped 7 previous similar messages
[17603.970607] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961917849c0 x1818616222030464/t0(0) o3->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 0 dl 1734374211 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'dd.0' uid:0 gid:0
[17603.974907] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 3 previous similar messages
[17658.773869] LustreError: 429911:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff896228103000: too many resent retries for object: 9663685440:8793: rc = -5
[17658.776352] LustreError: 429911:0:(osc_request.c:2602:brw_interpret()) Skipped 3 previous similar messages
[17659.829853] LustreError: 429914:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff896228103000: too many resent retries for object: 9663685440:8793: rc = -5
[17659.832414] LustreError: 429914:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[17671.318030] LustreError: lustre-OST0001-osc-ffff896228103000: operation ost_write to node 10.240.45.50@tcp failed: rc = -5
[17671.320039] LustreError: Skipped 59 previous similar messages
[17671.321116] LustreError: 429914:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff896197338d00 x1818616222054272/t0(0) o4->lustre-OST0001-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/224 e 0 to 0 dl 1734374278 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_01.0' uid:0 gid:0
[17671.325598] LustreError: 429914:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 55 previous similar messages
[17717.398205] LustreError: 429914:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff896228103000: too many resent retries for object: 10737421267:18345: rc = -5
[17717.400762] LustreError: 429914:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[17821.129406] LustreError: lustre-OST0001-osc-ffff896228103000: operation ost_read to node 10.240.45.50@tcp failed: rc = -5
[17821.131468] LustreError: Skipped 27 previous similar messages
[17821.132544] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff89619733a3c0 x1818616222096768/t0(0) o3->lustre-OST0001-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/440 e 0 to 0 dl 1734374480 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'dd.0' uid:0 gid:0
[17821.136873] LustreError: 429913:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 23 previous similar messages
[17877.143051] LustreError: 429911:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff896228103000: too many resent retries for object: 10737421267:18344: rc = -5
[17877.145605] LustreError: 429911:0:(osc_request.c:2602:brw_interpret()) Skipped 4 previous similar messages
[17880.113965] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398n: test append with parallel DIO ======= 18:40:34 \(1734374434\)
[17880.458329] Lustre: DEBUG MARKER: == sanity test 398n: test append with parallel DIO ======= 18:40:34 (1734374434)
[17902.464101] Autotest: Test running for 295 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[17917.075710] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398o: right kms with DIO ================== 18:41:11 \(1734374471\)
[17917.426115] Lustre: DEBUG MARKER: == sanity test 398o: right kms with DIO ================== 18:41:11 (1734374471)
[17937.421214] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398p: race aio with buffered i\/o ========== 18:41:31 \(1734374491\)
[17937.765245] Lustre: DEBUG MARKER: == sanity test 398p: race aio with buffered i/o ========== 18:41:31 (1734374491)
[18069.974128] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398q: race dio with buffered i\/o ========== 18:43:44 \(1734374624\)
[18070.309031] Lustre: DEBUG MARKER: == sanity test 398q: race dio with buffered i/o ========== 18:43:44 (1734374624)
[18203.277172] Autotest: Test running for 300 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[18438.900601] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398r: i\/o error on file read ============== 18:49:52 \(1734374992\)
[18439.269937] Lustre: DEBUG MARKER: == sanity test 398r: i/o error on file read ============== 18:49:52 (1734374992)
[18439.985189] LustreError: lustre-OST0000-osc-ffff896228103000: operation ost_read to node 10.240.45.50@tcp failed: rc = -5
[18439.987133] LustreError: Skipped 43 previous similar messages
[18439.988145] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5 req@ffff8961d48fe700 x1818616227887232/t0(0) o3->lustre-OST0000-osc-ffff896228103000@10.240.45.50@tcp:6/4 lens 488/4536 e 0 to 0 dl 1734375012 ref 2 fl Interpret:RMQU/200/0 rc -5/-5 job:'cat.0' uid:0 gid:0
[18439.992485] LustreError: 429912:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 39 previous similar messages
[18496.408172] LustreError: 429912:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff896228103000: too many resent retries for object: 9663685440:8797: rc = -5
[18496.410799] LustreError: 429912:0:(osc_request.c:2602:brw_interpret()) Skipped 2 previous similar messages
[18498.253664] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398s: i\/o error on mirror file read ======= 18:50:52 \(1734375052\)
[18498.610735] Lustre: DEBUG MARKER: == sanity test 398s: i/o error on mirror file read ======= 18:50:52 (1734375052)
[18501.087854] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 399a: fake write should not be slower than normal write ========================================================== 18:50:55 \(1734375055\)
[18501.442745] Lustre: DEBUG MARKER: == sanity test 399a: fake write should not be slower than normal write ========================================================== 18:50:55 (1734375055)
[18504.164401] Autotest: Test running for 305 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[18805.180868] Autotest: Test running for 310 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[18967.125357] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 399b: fake read should not be slower than normal read ========================================================== 18:58:41 \(1734375521\)
[18967.479064] Lustre: DEBUG MARKER: == sanity test 399b: fake read should not be slower than normal read ========================================================== 18:58:41 (1734375521)
[18985.360594] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 400a: Lustre client api program can compile and link ========================================================== 18:58:59 \(1734375539\)
[18985.711420] Lustre: DEBUG MARKER: == sanity test 400a: Lustre client api program can compile and link ========================================================== 18:58:59 (1734375539)
[18988.416029] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 400b: packaged headers can be compiled ==== 18:59:02 \(1734375542\)
[18988.754852] Lustre: DEBUG MARKER: == sanity test 400b: packaged headers can be compiled ==== 18:59:02 (1734375542)
[18990.871394] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401a: Verify if \'lctl list_param -R\' can list parameters recursively ========================================================== 18:59:04 \(1734375544\)
[18991.221038] Lustre: DEBUG MARKER: == sanity test 401a: Verify if 'lctl list_param -R' can list parameters recursively ========================================================== 18:59:04 (1734375544)
[18993.096811] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401aa: Verify that \'lctl list_param -p\' lists the correct path names ========================================================== 18:59:07 \(1734375547\)
[18993.443495] Lustre: DEBUG MARKER: == sanity test 401aa: Verify that 'lctl list_param -p' lists the correct path names ========================================================== 18:59:07 (1734375547)
[18995.275304] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401b: Verify \'lctl {get,set}_param\' continue after error ========================================================== 18:59:09 \(1734375549\)
[18995.624300] Lustre: DEBUG MARKER: == sanity test 401b: Verify 'lctl get_param' set_param' continue after error ========================================================== 18:59:09 (1734375549)
[18997.470042] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401c: Verify \'lctl set_param\' without value fails in either format. ========================================================== 18:59:11 \(1734375551\)
[18997.817427] Lustre: DEBUG MARKER: == sanity test 401c: Verify 'lctl set_param' without value fails in either format. ========================================================== 18:59:11 (1734375551)
[18999.646274] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401d: Verify \'lctl set_param\' accepts values containing \'=\' ========================================================== 18:59:13 \(1734375553\)
[19000.015227] Lustre: DEBUG MARKER: == sanity test 401d: Verify 'lctl set_param' accepts values containing '=' ========================================================== 18:59:13 (1734375553)
[19001.884717] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401db: Verify \'lctl set_param\' does not add trailing \'=\' ========================================================== 18:59:15 \(1734375555\)
[19002.229702] Lustre: DEBUG MARKER: == sanity test 401db: Verify 'lctl set_param' does not add trailing '=' ========================================================== 18:59:15 (1734375555)
[19002.732543] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19003.750617] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19004.768788] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19005.148203] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19006.166254] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19007.184930] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19008.203137] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19009.221661] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19010.239680] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19011.257761] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19012.275927] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19013.294316] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19014.312248] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19015.330249] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19016.348457] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19017.366524] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19018.384571] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19019.403024] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19020.422006] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19021.440258] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19022.458306] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19023.476083] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19024.493855] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19025.512464] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19026.530901] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19027.549116] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19028.567097] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19029.585453] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19030.603297] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19031.621640] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19032.639812] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19033.658230] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19034.676432] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19035.695526] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19036.713811] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19037.732036] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19038.750172] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19039.768384] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19040.787447] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19041.806071] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19042.823686] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19043.841177] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19044.858698] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19045.876844] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19046.897824] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19047.915697] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19048.933781] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19049.951737] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19050.969668] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19051.987728] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19053.005597] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19054.022293] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19055.039322] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19056.057488] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19057.075421] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19058.093065] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19059.109566] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19060.126442] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19061.144359] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19062.161274] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19063.177398] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19064.194532] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19065.212625] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19066.229668] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19067.245892] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19068.263295] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19069.281812] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19070.299030] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19071.316847] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19072.336423] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19073.354563] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19074.372730] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19075.390884] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19076.408984] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19077.427199] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19078.445245] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19079.463157] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19080.481425] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19081.499388] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19082.517684] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19083.535874] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19084.554369] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19085.572222] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19086.591322] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19087.609993] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19088.627558] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19089.645253] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19090.664163] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19091.683713] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19092.701946] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19093.720099] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19094.738185] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19097.601210] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19098.619179] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19099.637150] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[19099.973494] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401e: verify \'lctl get_param\' works with NID in parameter ========================================================== 19:00:54 \(1734375654\)
[19100.305437] Lustre: DEBUG MARKER: == sanity test 401e: verify 'lctl get_param' works with NID in parameter ========================================================== 19:00:54 (1734375654)
[19102.125852] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401f: check \'lctl list_param\' doesn\'t follow symlinks with --no-links ========================================================== 19:00:56 \(1734375656\)
[19102.477022] Lustre: DEBUG MARKER: == sanity test 401f: check 'lctl list_param' doesn't follow symlinks with --no-links ========================================================== 19:00:56 (1734375656)
[19104.321939] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 402: Return ENOENT to lod_generate_and_set_lovea ========================================================== 19:00:58 \(1734375658\)
[19104.673906] Lustre: DEBUG MARKER: == sanity test 402: Return ENOENT to lod_generate_and_set_lovea ========================================================== 19:00:58 (1734375658)
[19106.243566] Autotest: Test running for 315 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[19107.051600] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 403: i_nlink should not drop to zero due to aliasing ========================================================== 19:01:01 \(1734375661\)
[19107.399917] Lustre: DEBUG MARKER: == sanity test 403: i_nlink should not drop to zero due to aliasing ========================================================== 19:01:01 (1734375661)
[19107.692417] sysctl (727059): drop_caches: 2
[19110.153141] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 404: validate manual {de}activated works properly for OSPs ========================================================== 19:01:04 \(1734375664\)
[19110.501060] Lustre: DEBUG MARKER: == sanity test 404: validate manual {de}activated works properly for OSPs ========================================================== 19:01:04 (1734375664)
[19122.927100] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 405: Various layout swap lock tests ======= 19:01:16 \(1734375676\)
[19123.281625] Lustre: DEBUG MARKER: == sanity test 405: Various layout swap lock tests ======= 19:01:16 (1734375676)
[19130.082576] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_405 layout swap does not support DOM files so far
[19130.436105] Lustre: DEBUG MARKER: SKIP: sanity test_405 layout swap does not support DOM files so far
[19131.137873] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 406: DNE support fs default striping ====== 19:01:25 \(1734375685\)
[19131.493832] Lustre: DEBUG MARKER: == sanity test 406: DNE support fs default striping ====== 19:01:25 (1734375685)
[19131.907261] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_406 needs \>= 2 MDTs
[19132.260332] Lustre: DEBUG MARKER: SKIP: sanity test_406 needs >= 2 MDTs
[19132.702786] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_407 skipping ALWAYS excluded test 407
[19133.047255] Lustre: DEBUG MARKER: SKIP: sanity test_407 skipping ALWAYS excluded test 407
[19133.498667] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 408: drop_caches should not hang due to page leaks ========================================================== 19:01:27 \(1734375687\)
[19133.855053] Lustre: DEBUG MARKER: == sanity test 408: drop_caches should not hang due to page leaks ========================================================== 19:01:27 (1734375687)
[19134.117528] Lustre: *** cfs_fail_loc=40a, val=0***
[19134.118456] LustreError: 429911:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0000-osc-ffff896228103000: prep_req failed: rc = -22
[19134.120431] LustreError: 429911:0:(osc_cache.c:2153:osc_check_rpcs()) Read request failed with -22
[19135.848250] bash (728823): drop_caches: 2
[19137.591439] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 409: Large amount of cross-MDTs hard links on the same file ========================================================== 19:01:31 \(1734375691\)
[19137.944302] Lustre: DEBUG MARKER: == sanity test 409: Large amount of cross-MDTs hard links on the same file ========================================================== 19:01:31 (1734375691)
[19138.361330] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_409 needs \>= 2 MDTs
[19138.704699] Lustre: DEBUG MARKER: SKIP: sanity test_409 needs >= 2 MDTs
[19139.193942] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 410: Test inode number returned from kernel thread ========================================================== 19:01:33 \(1734375693\)
[19139.542969] Lustre: DEBUG MARKER: == sanity test 410: Test inode number returned from kernel thread ========================================================== 19:01:33 (1734375693)
[19140.199835] lustre_kinode_16832: CONFIG_X86_X32 is set
[19140.203433] lustre_kinode_16832: inode is 144115708588985662
[19140.204769] lustre_kinode_16832: inode is 144115708588985662
[19140.205919] lustre_kinode_16832: inode numbers are identical: 144115708588985662
[19141.974627] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 411a: Slab allocation error with cgroup does not LBUG ========================================================== 19:01:36 \(1734375696\)
[19142.326295] Lustre: DEBUG MARKER: == sanity test 411a: Slab allocation error with cgroup does not LBUG ========================================================== 19:01:36 (1734375696)
[19142.749518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_411a no setup for cgroup
[19143.083934] Lustre: DEBUG MARKER: SKIP: sanity test_411a no setup for cgroup
[19143.529538] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 411b: confirm Lustre can avoid OOM with reasonable cgroups limits ========================================================== 19:01:37 \(1734375697\)
[19143.882343] Lustre: DEBUG MARKER: == sanity test 411b: confirm Lustre can avoid OOM with reasonable cgroups limits ========================================================== 19:01:37 (1734375697)
[19144.299366] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_411b no setup for cgroup
[19144.648319] Lustre: DEBUG MARKER: SKIP: sanity test_411b no setup for cgroup
[19145.113265] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 412: mkdir on specific MDTs =============== 19:01:39 \(1734375699\)
[19145.485718] Lustre: DEBUG MARKER: == sanity test 412: mkdir on specific MDTs =============== 19:01:39 (1734375699)
[19145.902527] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_412 needs \>= 2 MDTs
[19146.239281] Lustre: DEBUG MARKER: SKIP: sanity test_412 needs >= 2 MDTs
[19146.698113] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413a: QoS mkdir with \'lfs mkdir -i -1\' ==== 19:01:40 \(1734375700\)
[19147.049094] Lustre: DEBUG MARKER: == sanity test 413a: QoS mkdir with 'lfs mkdir -i -1' ==== 19:01:40 (1734375700)
[19147.464998] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413a We need at least 2 MDTs for this test
[19147.830106] Lustre: DEBUG MARKER: SKIP: sanity test_413a We need at least 2 MDTs for this test
[19148.293320] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413b: QoS mkdir under dir whose default LMV starting MDT offset is -1 ========================================================== 19:01:42 \(1734375702\)
[19148.650010] Lustre: DEBUG MARKER: == sanity test 413b: QoS mkdir under dir whose default LMV starting MDT offset is -1 ========================================================== 19:01:42 (1734375702)
[19149.063094] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413b We need at least 2 MDTs for this test
[19149.416915] Lustre: DEBUG MARKER: SKIP: sanity test_413b We need at least 2 MDTs for this test
[19149.910505] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413c: mkdir with default LMV max inherit rr ========================================================== 19:01:43 \(1734375703\)
[19150.261770] Lustre: DEBUG MARKER: == sanity test 413c: mkdir with default LMV max inherit rr ========================================================== 19:01:43 (1734375703)
[19150.670795] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413c We need at least 2 MDTs for this test
[19151.017225] Lustre: DEBUG MARKER: SKIP: sanity test_413c We need at least 2 MDTs for this test
[19151.505358] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413d: inherit ROOT default LMV ============ 19:01:45 \(1734375705\)
[19151.857736] Lustre: DEBUG MARKER: == sanity test 413d: inherit ROOT default LMV ============ 19:01:45 (1734375705)
[19152.271561] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413d We need at least 2 MDTs for this test
[19152.624738] Lustre: DEBUG MARKER: SKIP: sanity test_413d We need at least 2 MDTs for this test
[19153.115224] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413e: check default max-inherit value ===== 19:01:47 \(1734375707\)
[19153.462544] Lustre: DEBUG MARKER: == sanity test 413e: check default max-inherit value ===== 19:01:47 (1734375707)
[19153.901209] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413e We need at least 2 MDTs for this test
[19154.235781] Lustre: DEBUG MARKER: SKIP: sanity test_413e We need at least 2 MDTs for this test
[19154.685226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413f: lfs getdirstripe -D list ROOT default LMV if it\'s not set on dir ========================================================== 19:01:48 \(1734375708\)
[19155.045353] Lustre: DEBUG MARKER: == sanity test 413f: lfs getdirstripe -D list ROOT default LMV if it's not set on dir ========================================================== 19:01:48 (1734375708)
[19155.460336] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413f We need at least 2 MDTs for this test
[19155.811359] Lustre: DEBUG MARKER: SKIP: sanity test_413f We need at least 2 MDTs for this test
[19156.285429] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413g: enforce ROOT default LMV on subdir mount ========================================================== 19:01:50 \(1734375710\)
[19156.636107] Lustre: DEBUG MARKER: == sanity test 413g: enforce ROOT default LMV on subdir mount ========================================================== 19:01:50 (1734375710)
[19157.053394] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413g We need at least 2 MDTs for this test
[19157.400269] Lustre: DEBUG MARKER: SKIP: sanity test_413g We need at least 2 MDTs for this test
[19157.860850] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413h: don\'t stick to parent for round-robin dirs ========================================================== 19:01:51 \(1734375711\)
[19158.215958] Lustre: DEBUG MARKER: == sanity test 413h: don't stick to parent for round-robin dirs ========================================================== 19:01:51 (1734375711)
[19158.632850] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413h We need at least 2 MDTs for this test
[19158.986399] Lustre: DEBUG MARKER: SKIP: sanity test_413h We need at least 2 MDTs for this test
[19159.436604] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413i: check default layout inheritance ==== 19:01:53 \(1734375713\)
[19159.802717] Lustre: DEBUG MARKER: == sanity test 413i: check default layout inheritance ==== 19:01:53 (1734375713)
[19160.215076] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413i needs \>= 2 MDTs
[19160.570649] Lustre: DEBUG MARKER: SKIP: sanity test_413i needs >= 2 MDTs
[19160.999079] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413j: set default LMV by setxattr ========= 19:01:55 \(1734375715\)
[19161.353486] Lustre: DEBUG MARKER: == sanity test 413j: set default LMV by setxattr ========= 19:01:55 (1734375715)
[19161.760026] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_413j needs \>= 2 MDTs
[19162.107675] Lustre: DEBUG MARKER: SKIP: sanity test_413j needs >= 2 MDTs
[19162.554900] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413k: QoS mkdir exclude prefixes ========== 19:01:56 \(1734375716\)
[19162.902906] Lustre: DEBUG MARKER: == sanity test 413k: QoS mkdir exclude prefixes ========== 19:01:56 (1734375716)
[19165.000109] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413z: 413 test cleanup ==================== 19:01:59 \(1734375719\)
[19165.347220] Lustre: DEBUG MARKER: == sanity test 413z: 413 test cleanup ==================== 19:01:59 (1734375719)
[19167.191538] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 414: simulate ENOMEM in ptlrpc_register_bulk\(\) ========================================================== 19:02:01 \(1734375721\)
[19167.541153] Lustre: DEBUG MARKER: == sanity test 414: simulate ENOMEM in ptlrpc_register_bulk() ========================================================== 19:02:01 (1734375721)
[19167.754776] Lustre: *** cfs_fail_loc=521, val=0***
[19167.755661] LustreError: 429911:0:(niobuf.c:389:ptlrpc_register_bulk()) lustre-OST0005-osc-ffff896228103000: LNetMEAttach failed x1818616229529473/1: rc = -12
[19168.023706] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[19168.037752] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[19168.196676] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[19168.260868] Lustre: Unmounted lustre-client
[19168.274268] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[19168.287272] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[19168.361602] Lustre: Mounted lustre-client
[19170.097773] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 415: lock revoke is not missing =========== 19:02:04 \(1734375724\)
[19170.447260] Lustre: DEBUG MARKER: == sanity test 415: lock revoke is not missing =========== 19:02:04 (1734375724)
[19171.160231] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19172.498344] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19189.189263] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19190.239975] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19191.472685] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 416: transaction start failure won\'t cause system hung ========================================================== 19:02:25 \(1734375745\)
[19191.827051] Lustre: DEBUG MARKER: == sanity test 416: transaction start failure won't cause system hung ========================================================== 19:02:25 (1734375745)
[19194.018226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 417: disable remote dir, striped dir and dir migration ========================================================== 19:02:28 \(1734375748\)
[19194.366895] Lustre: DEBUG MARKER: == sanity test 417: disable remote dir, striped dir and dir migration ========================================================== 19:02:28 (1734375748)
[19194.777492] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_417 needs \>= 2 MDTs
[19195.131465] Lustre: DEBUG MARKER: SKIP: sanity test_417 needs >= 2 MDTs
[19195.585759] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 418: df and lfs df outputs match ========== 19:02:29 \(1734375749\)
[19195.948251] Lustre: DEBUG MARKER: == sanity test 418: df and lfs df outputs match ========== 19:02:29 (1734375749)
[19206.491037] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[19207.548251] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[19208.486800] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[19209.434138] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[19210.794771] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19213.247544] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19214.552581] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[19215.630506] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[19216.991451] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19219.259373] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19221.916312] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 419: Verify open file by name doesn\'t crash kernel ========================================================== 19:02:55 \(1734375775\)
[19222.300305] Lustre: DEBUG MARKER: == sanity test 419: Verify open file by name doesn't crash kernel ========================================================== 19:02:55 (1734375775)
[19224.324162] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 420: clear SGID bit on non-directories for non-members ========================================================== 19:02:58 \(1734375778\)
[19224.677669] Lustre: DEBUG MARKER: == sanity test 420: clear SGID bit on non-directories for non-members ========================================================== 19:02:58 (1734375778)
[19226.743044] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421a: simple rm by fid ==================== 19:03:00 \(1734375780\)
[19227.092727] Lustre: DEBUG MARKER: == sanity test 421a: simple rm by fid ==================== 19:03:00 (1734375780)
[19229.035057] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421b: rm by fid on open file ============== 19:03:03 \(1734375783\)
[19229.383719] Lustre: DEBUG MARKER: == sanity test 421b: rm by fid on open file ============== 19:03:03 (1734375783)
[19231.284629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421c: rm by fid against hardlinked files == 19:03:05 \(1734375785\)
[19231.631582] Lustre: DEBUG MARKER: == sanity test 421c: rm by fid against hardlinked files == 19:03:05 (1734375785)
[19232.383097] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19233.177705] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19236.072051] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421d: rmfid en masse ====================== 19:03:10 \(1734375790\)
[19236.425339] Lustre: DEBUG MARKER: == sanity test 421d: rmfid en masse ====================== 19:03:10 (1734375790)
[19237.162273] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19241.280724] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19251.507151] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421e: rmfid in DNE ======================== 19:03:25 \(1734375805\)
[19251.855613] Lustre: DEBUG MARKER: == sanity test 421e: rmfid in DNE ======================== 19:03:25 (1734375805)
[19252.268425] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_421e needs \>= 2 MDTs
[19252.642854] Lustre: DEBUG MARKER: SKIP: sanity test_421e needs >= 2 MDTs
[19253.112257] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421f: rmfid checks permissions ============ 19:03:27 \(1734375807\)
[19253.469520] Lustre: DEBUG MARKER: == sanity test 421f: rmfid checks permissions ============ 19:03:27 (1734375807)
[19253.931030] Lustre: DEBUG MARKER: mkdir -p /tmp/lustre-rS6TH4
[19253.943957] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,user_fid2path 10.240.38.186@tcp:/lustre /tmp/lustre-rS6TH4
[19254.001977] Lustre: Mounted lustre-client
[19255.563767] Lustre: DEBUG MARKER: grep -c /tmp/lustre-rS6TH4' ' /proc/mounts
[19255.577456] Lustre: DEBUG MARKER: lsof -t /tmp/lustre-rS6TH4
[19255.734469] Lustre: DEBUG MARKER: umount /tmp/lustre-rS6TH4 2>&1
[19255.779691] Lustre: Unmounted lustre-client
[19256.096097] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421g: rmfid to return errors properly ===== 19:03:30 \(1734375810\)
[19256.447071] Lustre: DEBUG MARKER: == sanity test 421g: rmfid to return errors properly ===== 19:03:30 (1734375810)
[19256.861953] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_421g needs \>= 2 MDTs
[19257.216170] Lustre: DEBUG MARKER: SKIP: sanity test_421g needs >= 2 MDTs
[19257.669669] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421h: rmfid with fileset mount ============ 19:03:31 \(1734375811\)
[19258.035999] Lustre: DEBUG MARKER: == sanity test 421h: rmfid with fileset mount ============ 19:03:31 (1734375811)
[19258.453968] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_other
[19258.466302] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre_other
[19258.520732] Lustre: Mounted lustre-client
[19258.529385] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[19258.544704] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_other//d421h.sanity/subdir
[19258.564613] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_other' ' /proc/mounts ||
[19258.564613] umount /mnt/lustre_other
[19258.612605] Lustre: Unmounted lustre-client
[19258.619550] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d421h.sanity/subdir /mnt/lustre_other
[19258.724875] Lustre: DEBUG MARKER: grep -c /mnt/lustre_other' ' /proc/mounts
[19258.740381] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_other
[19258.904086] Lustre: DEBUG MARKER: umount /mnt/lustre_other 2>&1
[19260.685194] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 422: kill a process with RPC in progress == 19:03:34 \(1734375814\)
[19261.046931] Lustre: DEBUG MARKER: == sanity test 422: kill a process with RPC in progress == 19:03:34 (1734375814)
[19261.241269] Lustre: DEBUG MARKER: lctl get_param -n at_max
[19261.649987] Lustre: DEBUG MARKER: lctl set_param at_max=0
[19283.609334] Lustre: 744662:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734375816/real 1734375816] req@ffff8961d459a080 x1818616235230080/t0(0) o101->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 576/1744 e 0 to 1 dl 1734375836 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0
[19283.614639] Lustre: lustre-MDT0000-mdc-ffff8961ce4ec000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[19283.617456] Lustre: Skipped 6 previous similar messages
[19283.622026] Lustre: lustre-MDT0000-mdc-ffff8961ce4ec000: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[19304.089933] Lustre: 744675:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734375837/real 1734375837] req@ffff8961d4599040 x1818616235231872/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:13/10 lens 504/920 e 0 to 1 dl 1734375857 ref 2 fl Rpc:XQr/202/ffffffff rc -11/-1 job:'mv.0' uid:0 gid:0
[19304.095277] Lustre: 744675:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[19305.815700] Lustre: DEBUG MARKER: touch
[19324.569383] Lustre: 744662:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734375858/real 1734375858] req@ffff8961d459a080 x1818616235230080/t0(0) o101->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 576/1744 e 0 to 1 dl 1734375878 ref 2 fl Rpc:XQr/202/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0
[19324.574749] Lustre: lustre-MDT0000-mdc-ffff8961ce4ec000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[19324.577568] Lustre: Skipped 1 previous similar message
[19324.581827] Lustre: lustre-MDT0000-mdc-ffff8961ce4ec000: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[19324.583657] Lustre: Skipped 1 previous similar message
[19324.644328] Lustre: DEBUG MARKER: lctl set_param at_max=600
[19327.407744] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 423: statfs should return a right data ==== 19:04:41 \(1734375881\)
[19327.751913] Lustre: DEBUG MARKER: == sanity test 423: statfs should return a right data ==== 19:04:41 (1734375881)
[19331.583714] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 424: simulate ENOMEM in ptl_send_rpc bulk reply ME attach ========================================================== 19:04:45 \(1734375885\)
[19331.928739] Lustre: DEBUG MARKER: == sanity test 424: simulate ENOMEM in ptl_send_rpc bulk reply ME attach ========================================================== 19:04:45 (1734375885)
[19332.139386] Lustre: *** cfs_fail_loc=522, val=0***
[19332.140347] LustreError: 429911:0:(niobuf.c:899:ptl_send_rpc()) LNetMEAttach failed: -12
[19334.308417] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 425: lock count should not exceed lru size ========================================================== 19:04:48 \(1734375888\)
[19334.658921] Lustre: DEBUG MARKER: == sanity test 425: lock count should not exceed lru size ========================================================== 19:04:48 (1734375888)
[19342.841072] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 426: splice test on Lustre ================ 19:04:56 \(1734375896\)
[19343.207023] Lustre: DEBUG MARKER: == sanity test 426: splice test on Lustre ================ 19:04:56 (1734375896)
[19345.687220] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 427: Failed DNE2 update request shouldn\'t corrupt updatelog ========================================================== 19:04:59 \(1734375899\)
[19346.028949] Lustre: DEBUG MARKER: == sanity test 427: Failed DNE2 update request shouldn't corrupt updatelog ========================================================== 19:04:59 (1734375899)
[19346.447178] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_427 needs \>= 2 MDTs
[19346.803179] Lustre: DEBUG MARKER: SKIP: sanity test_427 needs >= 2 MDTs
[19347.252974] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 428: large block size IO should not hang == 19:05:01 \(1734375901\)
[19347.601514] Lustre: DEBUG MARKER: == sanity test 428: large block size IO should not hang == 19:05:01 (1734375901)
[19408.356666] Autotest: Test running for 320 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[19450.155287] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 429: verify if opencache flag on client side does work ========================================================== 19:06:44 \(1734376004\)
[19450.515305] Lustre: DEBUG MARKER: == sanity test 429: verify if opencache flag on client side does work ========================================================== 19:06:44 (1734376004)
[19452.449935] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 430a: lseek: SEEK_DATA\/SEEK_HOLE basic functionality ========================================================== 19:06:46 \(1734376006\)
[19452.800603] Lustre: DEBUG MARKER: == sanity test 430a: lseek: SEEK_DATA/SEEK_HOLE basic functionality ========================================================== 19:06:46 (1734376006)
[19456.222368] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 430b: lseek: SEEK_DATA\/SEEK_HOLE special cases ========================================================== 19:06:50 \(1734376010\)
[19456.576666] Lustre: DEBUG MARKER: == sanity test 430b: lseek: SEEK_DATA/SEEK_HOLE special cases ========================================================== 19:06:50 (1734376010)
[19459.288732] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 430c: lseek: external tools check ========= 19:06:53 \(1734376013\)
[19459.644904] Lustre: DEBUG MARKER: == sanity test 430c: lseek: external tools check ========= 19:06:53 (1734376013)
[19461.655701] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 431: Restart transaction for IO =========== 19:06:55 \(1734376015\)
[19462.009067] Lustre: DEBUG MARKER: == sanity test 431: Restart transaction for IO =========== 19:06:55 (1734376015)
[19464.539701] bash (749775): drop_caches: 3
[19466.981663] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 432: mv dir from outside Lustre =========== 19:07:00 \(1734376020\)
[19467.362317] Lustre: DEBUG MARKER: == sanity test 432: mv dir from outside Lustre =========== 19:07:00 (1734376020)
[19480.079875] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 433: ldlm lock cancel releases dentries and inodes ========================================================== 19:07:14 \(1734376034\)
[19480.419571] Lustre: DEBUG MARKER: == sanity test 433: ldlm lock cancel releases dentries and inodes ========================================================== 19:07:14 (1734376034)
[19481.162613] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19481.912037] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19483.420207] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[19484.167271] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19489.094911] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 434: Client should not send RPCs for security.selinux with SElinux disabled ========================================================== 19:07:23 \(1734376043\)
[19489.472434] Lustre: DEBUG MARKER: == sanity test 434: Client should not send RPCs for security.selinux with SElinux disabled ========================================================== 19:07:23 (1734376043)
[19490.764212] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[19490.764212] [[ -z "" ]] && param= ||
[19490.764212] param=$(grep <<< "$params");
[19490.764212] [[ -z $param ]] && param="$params";
[19490.764212] while read s; do echo client $s;
[19490.764212] done <<< "$param"
[19492.874350] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff8961ce4ec000.xattr_cache=1
[19493.213235] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 440: bash completion for lfs, lctl ======== 19:07:27 \(1734376047\)
[19493.556207] Lustre: DEBUG MARKER: == sanity test 440: bash completion for lfs, lctl ======== 19:07:27 (1734376047)
[19495.451207] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 442: truncate vs read\/write should not panic ========================================================== 19:07:29 \(1734376049\)
[19495.801198] Lustre: DEBUG MARKER: == sanity test 442: truncate vs read/write should not panic ========================================================== 19:07:29 (1734376049)
[19496.963534] LustreError: 753760:0:(llite_lib.c:3120:ll_truncate_inode_pages_final()) cfs_fail_timeout id 1430 sleeping for 5000ms
[19502.057758] LustreError: 753760:0:(llite_lib.c:3120:ll_truncate_inode_pages_final()) cfs_fail_timeout id 1430 awake
[19503.765039] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 460d: Check encrypt pools output ========== 19:07:37 \(1734376057\)
[19504.116003] Lustre: DEBUG MARKER: == sanity test 460d: Check encrypt pools output ========== 19:07:37 (1734376057)
[19506.163061] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600a: basic test for mlock\(\)ed file ======= 19:07:40 \(1734376060\)
[19506.504286] Lustre: DEBUG MARKER: == sanity test 600a: basic test for mlock()ed file ======= 19:07:40 (1734376060)
[19509.438270] bash (754534): drop_caches: 1
[19512.553575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600b: mlock a file \(via vmtouch\) larger than max_cached_mb ========================================================== 19:07:46 \(1734376066\)
[19512.897244] Lustre: DEBUG MARKER: == sanity test 600b: mlock a file (via vmtouch) larger than max_cached_mb ========================================================== 19:07:46 (1734376066)
[19522.825787] bash (755044): drop_caches: 1
[19524.999550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600c: Test I\/O when mlocked page count \> @max_cached_mb ========================================================== 19:07:59 \(1734376079\)
[19525.357327] Lustre: DEBUG MARKER: == sanity test 600c: Test I/O when mlocked page count > @max_cached_mb ========================================================== 19:07:59 (1734376079)
[19531.334850] bash (755542): drop_caches: 3
[19532.970064] bash (755542): drop_caches: 1
[19534.724829] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600d: Test I\/O with limited LRU page slots \(some was mlocked\) ========================================================== 19:08:08 \(1734376088\)
[19535.065692] Lustre: DEBUG MARKER: == sanity test 600d: Test I/O with limited LRU page slots (some was mlocked) ========================================================== 19:08:08 (1734376088)
[19538.501536] bash (756059): drop_caches: 1
[19542.546255] bash (756059): drop_caches: 1
[19544.820954] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 801a: write barrier user interfaces and stat machine ========================================================== 19:08:18 \(1734376098\)
[19545.179944] Lustre: DEBUG MARKER: == sanity test 801a: write barrier user interfaces and stat machine ========================================================== 19:08:18 (1734376098)
[19545.967866] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 debug_mb=150
[19576.953239] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug_mb=11
[19577.686314] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19580.306867] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 801b: modification will be blocked by write barrier ========================================================== 19:08:54 \(1734376134\)
[19580.655954] Lustre: DEBUG MARKER: == sanity test 801b: modification will be blocked by write barrier ========================================================== 19:08:54 (1734376134)
[19581.415274] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 debug_mb=150
[19582.143239] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[19585.275413] Lustre: 758154:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS req@ffff8961d555b0c0 x1818616258983680/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 488/456 e 0 to 0 dl 1734376194 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'rm.0' uid:0 gid:0
[19585.286064] Lustre: 758154:0:(client.c:1532:after_reply()) Skipped 1 previous similar message
[19594.831991] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug_mb=11
[19597.252240] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 801c: rescan barrier bitmap =============== 19:09:11 \(1734376151\)
[19597.604518] Lustre: DEBUG MARKER: == sanity test 801c: rescan barrier bitmap =============== 19:09:11 (1734376151)
[19598.009020] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_801c needs \>= 2 MDTs
[19598.383719] Lustre: DEBUG MARKER: SKIP: sanity test_801c needs >= 2 MDTs
[19598.844343] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 802b: be able to set MDTs to readonly ===== 19:09:12 \(1734376152\)
[19599.183801] Lustre: DEBUG MARKER: == sanity test 802b: be able to set MDTs to readonly ===== 19:09:12 (1734376152)
[19604.439309] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 802c: be able to set OFDs to readonly ===== 19:09:18 \(1734376158\)
[19604.787234] Lustre: DEBUG MARKER: == sanity test 802c: be able to set OFDs to readonly ===== 19:09:18 (1734376158)
[19609.092567] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 803a: verify agent object for remote object ========================================================== 19:09:23 \(1734376163\)
[19609.461470] Lustre: DEBUG MARKER: == sanity test 803a: verify agent object for remote object ========================================================== 19:09:23 (1734376163)
[19609.886404] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_803a needs \>= 2 MDTs
[19610.223954] Lustre: DEBUG MARKER: SKIP: sanity test_803a needs >= 2 MDTs
[19610.676541] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 803b: remote object can getattr from cache ========================================================== 19:09:24 \(1734376164\)
[19611.021311] Lustre: DEBUG MARKER: == sanity test 803b: remote object can getattr from cache ========================================================== 19:09:24 (1734376164)
[19611.437354] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_803b needs \>= 2 MDTs
[19611.785815] Lustre: DEBUG MARKER: SKIP: sanity test_803b needs >= 2 MDTs
[19612.244973] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 804: verify agent entry for remote entry == 19:09:26 \(1734376166\)
[19612.607840] Lustre: DEBUG MARKER: == sanity test 804: verify agent entry for remote entry == 19:09:26 (1734376166)
[19613.002470] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_804 needs \>= 2 MDTs
[19613.344708] Lustre: DEBUG MARKER: SKIP: sanity test_804 needs >= 2 MDTs
[19613.793437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 805: ZFS can remove from full fs ========== 19:09:27 \(1734376167\)
[19614.135869] Lustre: DEBUG MARKER: == sanity test 805: ZFS can remove from full fs ========== 19:09:27 (1734376167)
[19614.892671] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_805 ZFS specific test
[19615.237999] Lustre: DEBUG MARKER: SKIP: sanity test_805 ZFS specific test
[19615.677248] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 806: Verify Lazy Size on MDS ============== 19:09:29 \(1734376169\)
[19616.019642] Lustre: DEBUG MARKER: == sanity test 806: Verify Lazy Size on MDS ============== 19:09:29 (1734376169)
[19625.706078] Lustre: DEBUG MARKER: multiop /mnt/lustre/f806.sanity Oz0w1048576c
[19636.530889] Lustre: DEBUG MARKER: multiop /mnt/lustre/f806.sanity Oz1048576w1048576c
[19643.684183] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 807a: verify LSOM syncing tool ============ 19:09:57 \(1734376197\)
[19644.034624] Lustre: DEBUG MARKER: == sanity test 807a: verify LSOM syncing tool ============ 19:09:57 (1734376197)
[19646.119594] Lustre: DEBUG MARKER: multiop /mnt/lustre/f807a.sanity Oz0w1048576c
[19647.061640] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19648.357647] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks osc
[19648.836520] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks osc
[19649.425740] Lustre: DEBUG MARKER: sync ; sleep 5 ; sync
[19700.260929] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 807b: verify lfs somsync utility ========== 19:10:54 \(1734376254\)
[19700.622781] Lustre: DEBUG MARKER: == sanity test 807b: verify lfs somsync utility ========== 19:10:54 (1734376254)
[19701.674220] Lustre: DEBUG MARKER: multiop /mnt/lustre/f807b.sanity Oz0w1048576c
[19702.606589] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19703.732012] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks osc
[19704.086343] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks osc
[19704.885879] Lustre: DEBUG MARKER: sync ; sleep 5 ; sync
[19709.934827] Autotest: Test running for 325 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[19791.169137] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 808: Check trusted.som xattr not logged in Changelogs ========================================================== 19:12:25 \(1734376345\)
[19791.519041] Lustre: DEBUG MARKER: == sanity test 808: Check trusted.som xattr not logged in Changelogs ========================================================== 19:12:25 (1734376345)
[19796.195831] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 809: Verify no SOM xattr store for DoM-only files ========================================================== 19:12:30 \(1734376350\)
[19796.571482] Lustre: DEBUG MARKER: == sanity test 809: Verify no SOM xattr store for DoM-only files ========================================================== 19:12:30 (1734376350)
[19798.518135] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 810: partial page writes on ZFS \(LU-11663\) ========================================================== 19:12:32 \(1734376352\)
[19798.871281] Lustre: DEBUG MARKER: == sanity test 810: partial page writes on ZFS (LU-11663) ========================================================== 19:12:32 (1734376352)
[19799.033493] Lustre: *** cfs_fail_loc=411, val=0***
[19799.034407] Lustre: Skipped 1 previous similar message
[19799.642412] Lustre: *** cfs_fail_loc=411, val=0***
[19799.643379] Lustre: Skipped 1 previous similar message
[19801.029307] Lustre: *** cfs_fail_loc=411, val=0***
[19801.030259] Lustre: Skipped 5 previous similar messages
[19803.040634] Lustre: *** cfs_fail_loc=411, val=0***
[19803.041608] Lustre: Skipped 11 previous similar messages
[19807.730442] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[19808.109926] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[19808.562694] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 812a: do not drop reqs generated when imp is going to idle \(LU-11951\) ========================================================== 19:12:42 \(1734376362\)
[19808.902369] Lustre: DEBUG MARKER: == sanity test 812a: do not drop reqs generated when imp is going to idle (LU-11951) ========================================================== 19:12:42 (1734376362)
[19809.089374] Lustre: DEBUG MARKER: lctl get_param -n at_min
[19809.122998] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19809.854894] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19810.183626] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19810.581102] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[19810.920563] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[19811.442651] Lustre: DEBUG MARKER: lctl get_param -n at_min
[19811.477485] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19812.208378] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19812.540458] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19822.021154] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in CONNECTING state after 9 sec
[19822.346526] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in CONNECTING state after 9 sec
[19824.617052] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 812b: do not drop no resend request for idle connect ========================================================== 19:12:58 \(1734376378\)
[19824.984371] Lustre: DEBUG MARKER: == sanity test 812b: do not drop no resend request for idle connect ========================================================== 19:12:58 (1734376378)
[19825.147877] Lustre: DEBUG MARKER: lctl get_param -n at_min
[19825.181183] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19825.912264] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19826.240675] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19826.625853] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[19826.962815] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[19827.470910] Lustre: DEBUG MARKER: lctl get_param -n at_min
[19827.504553] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19828.232550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19828.577020] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19842.091793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in CONNECTING state after 13 sec
[19842.504098] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in CONNECTING state after 13 sec
[19843.120245] Lustre: DEBUG MARKER: lctl get_param -n at_min
[19843.159708] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19843.900107] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19844.223854] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19857.759840] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in IDLE state after 13 sec
[19858.182750] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in IDLE state after 13 sec
[19860.068835] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 812c: idle import vs lock enqueue race ==== 19:13:34 \(1734376414\)
[19860.424317] Lustre: DEBUG MARKER: == sanity test 812c: idle import vs lock enqueue race ==== 19:13:34 (1734376414)
[19860.607181] Lustre: DEBUG MARKER: lctl get_param -n at_min
[19860.639282] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19861.353270] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19861.678744] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[19862.068211] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[19862.415021] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[19872.410810] LustreError: 710490:0:(import.c:2047:ptlrpc_disconnect_and_idle_import()) cfs_race id 533 sleeping
[19877.530790] LustreError: 710490:0:(import.c:2047:ptlrpc_disconnect_and_idle_import()) cfs_fail_race id 533 awake: rc=0
[19877.563121] LustreError: 768791:0:(osc_lock.c:1017:osc_lock_enqueue()) cfs_fail_race id 533 waking
[19878.146832] LustreError: 768791:0:(osc_lock.c:1017:osc_lock_enqueue()) cfs_fail_race id 533 waking
[19880.508252] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 813: File heat verfication ================ 19:13:54 \(1734376434\)
[19880.835375] Lustre: DEBUG MARKER: == sanity test 813: File heat verfication ================ 19:13:54 (1734376434)
[20009.275755] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 814: sparse cp works as expected \(LU-12361\) ========================================================== 19:16:03 \(1734376563\)
[20009.635519] Lustre: DEBUG MARKER: == sanity test 814: sparse cp works as expected (LU-12361) ========================================================== 19:16:03 (1734376563)
[20011.052234] Autotest: Test running for 330 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[20011.685995] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 815: zero byte tiny write doesn\'t hang \(LU-12382\) ========================================================== 19:16:05 \(1734376565\)
[20012.027957] Lustre: DEBUG MARKER: == sanity test 815: zero byte tiny write doesn't hang (LU-12382) ========================================================== 19:16:05 (1734376565)
[20013.859372] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 816: do not reset lru_resize on idle reconnect ========================================================== 19:16:07 \(1734376567\)
[20014.206155] Lustre: DEBUG MARKER: == sanity test 816: do not reset lru_resize on idle reconnect ========================================================== 19:16:07 (1734376567)
[20014.389815] Lustre: DEBUG MARKER: lctl get_param -n at_min
[20014.421956] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20015.140677] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[20015.473360] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[20015.867358] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[20016.198312] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in FULL state after 0 sec
[20016.402052] Lustre: DEBUG MARKER: lctl get_param -n at_min
[20016.435181] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20017.164946] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[20017.509816] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid 50
[20027.056742] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in IDLE state after 9 sec
[20027.337302] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8961ce4ec000.ost_server_uuid in IDLE state after 9 sec
[20029.288125] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_817 skipping ALWAYS excluded test 817
[20029.635916] Lustre: DEBUG MARKER: SKIP: sanity test_817 skipping ALWAYS excluded test 817
[20030.088371] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 818: unlink with failed llog ============== 19:16:24 \(1734376584\)
[20030.423124] Lustre: DEBUG MARKER: == sanity test 818: unlink with failed llog ============== 19:16:24 (1734376584)
[20036.252292] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[20036.255082] Lustre: lustre-MDT0000-mdc-ffff8961ce4ec000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[20036.258412] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f577070d to 0x5d837553f580dda1
[20036.260891] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[20037.760568] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff8961d27f2a40 x1818616259120896/t68719519511(68719519511) o101->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 576/608 e 0 to 0 dl 1734376608 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'md5sum.0' uid:0 gid:0
[20037.765390] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 2 previous similar messages
[20056.733172] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[20056.739862] Lustre: lustre-MDT0000-mdc-ffff8961ce4ec000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[20056.742818] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f580dda1 to 0x5d837553f580e271
[20056.745561] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[20056.747183] Lustre: Skipped 1 previous similar message
[20056.769242] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff8961d27f2a40 x1818616259120896/t68719519511(68719519511) o101->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 576/608 e 0 to 0 dl 1734376627 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'md5sum.0' uid:0 gid:0
[20056.773943] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 4 previous similar messages
[20060.980382] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20062.124902] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[20062.486433] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[20062.651201] Lustre: 429913:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734376600/real 1734376600] req@ffff8961d27f2080 x1818616259313024/t0(0) o400->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734376616 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[20062.785354] Lustre: DEBUG MARKER: lctl get_param -n at_max
[20063.039791] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[20063.550219] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[20065.704346] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 819a: too big niobuf in read ============== 19:16:59 \(1734376619\)
[20066.052249] Lustre: DEBUG MARKER: == sanity test 819a: too big niobuf in read ============== 19:16:59 (1734376619)
[20066.971229] Lustre: 429914:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734376605/real 1734376605] req@ffff8961d27f0000 x1818616259313408/t0(0) o400->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 224/224 e 0 to 1 dl 1734376621 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[20069.170315] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 819b: too big niobuf in write ============= 19:17:03 \(1734376623\)
[20069.512956] Lustre: DEBUG MARKER: == sanity test 819b: too big niobuf in write ============= 19:17:03 (1734376623)
[20072.780228] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 820: update max EA from open intent ======= 19:17:06 \(1734376626\)
[20073.125110] Lustre: DEBUG MARKER: == sanity test 820: update max EA from open intent ======= 19:17:06 (1734376626)
[20073.559609] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_820 needs \>= 2 MDTs
[20073.891727] Lustre: DEBUG MARKER: SKIP: sanity test_820 needs >= 2 MDTs
[20074.339013] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 823: Setting create_count \> OST_MAX_PRECREATE is lowered to maximum ========================================================== 19:17:08 \(1734376628\)
[20074.706097] Lustre: DEBUG MARKER: == sanity test 823: Setting create_count > OST_MAX_PRECREATE is lowered to maximum ========================================================== 19:17:08 (1734376628)
[20077.258658] Lustre: DEBUG MARKER: /usr/sbin/lctl mark setting create_count to 100200:
[20077.613367] Lustre: DEBUG MARKER: setting create_count to 100200:
[20078.038526] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -result- count: 9984 with max: 20000, expecting: 9984
[20078.385952] Lustre: DEBUG MARKER: -result- count: 9984 with max: 20000, expecting: 9984
[20082.734531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 831: throttling unlink\/setattr queuing on OSP ========================================================== 19:17:16 \(1734376636\)
[20083.100209] Lustre: DEBUG MARKER: == sanity test 831: throttling unlink/setattr queuing on OSP ========================================================== 19:17:16 (1734376636)
[20085.273275] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[20087.226518] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[20088.718975] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[20090.270473] Lustre: 775954:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS req@ffff896220153400 x1818616259869184/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 488/456 e 0 to 0 dl 1734376660 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[20090.274959] Lustre: 775954:0:(client.c:1532:after_reply()) Skipped 1 previous similar message
[20096.411915] Lustre: 775954:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS req@ffff896229f0ed80 x1818616259870592/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 488/456 e 0 to 0 dl 1734376666 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[20099.488312] Lustre: 775954:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS req@ffff8961d27f3740 x1818616259897216/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 488/456 e 0 to 0 dl 1734376669 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[20105.628997] Lustre: 775954:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS req@ffff8961dbc80d00 x1818616259907200/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 488/456 e 0 to 0 dl 1734376675 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[20114.780057] Lustre: 775954:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS req@ffff89621c193a80 x1818616259935232/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 488/456 e 0 to 0 dl 1734376685 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[20114.784414] Lustre: 775954:0:(client.c:1532:after_reply()) Skipped 2 previous similar messages
[20136.379998] Lustre: 775954:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS req@ffff8961d27f2080 x1818616260014080/t0(0) o36->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 488/456 e 0 to 0 dl 1734376706 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[20136.384317] Lustre: 775954:0:(client.c:1532:after_reply()) Skipped 3 previous similar messages
[20170.293515] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[20179.424932] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 832: lfs rm_entry ========================= 19:18:53 \(1734376733\)
[20179.779092] Lustre: DEBUG MARKER: == sanity test 832: lfs rm_entry ========================= 19:18:53 (1734376733)
[20180.209544] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_832 needs \>= 2 MDTs
[20180.555535] Lustre: DEBUG MARKER: SKIP: sanity test_832 needs >= 2 MDTs
[20181.016663] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 833: Mixed buffered\/direct read and write should not return -EIO ========================================================== 19:18:55 \(1734376735\)
[20181.381071] Lustre: DEBUG MARKER: == sanity test 833: Mixed buffered/direct read and write should not return -EIO ========================================================== 19:18:55 (1734376735)
[20215.655650] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_842 skipping SLOW test 842
[20216.012777] Lustre: DEBUG MARKER: SKIP: sanity test_842 skipping SLOW test 842
[20216.449219] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 850: lljobstat can parse living and aggregated job_stats ========================================================== 19:19:30 \(1734376770\)
[20216.803213] Lustre: DEBUG MARKER: == sanity test 850: lljobstat can parse living and aggregated job_stats ========================================================== 19:19:30 (1734376770)
[20218.820315] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 851: fanotify can monitor open\/read\/write\/close events for lustre fs ========================================================== 19:19:32 \(1734376772\)
[20219.183399] Lustre: DEBUG MARKER: == sanity test 851: fanotify can monitor open/read/write/close events for lustre fs ========================================================== 19:19:32 (1734376772)
[20220.529870] Lustre: DEBUG MARKER: stat -c %s /tmp/report_test_851_15790
[20220.917612] Lustre: DEBUG MARKER: stat -c %s /tmp/report_test_851_15790
[20222.825208] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 852: mkdir using intent lock for striped directory ========================================================== 19:19:36 \(1734376776\)
[20223.169095] Lustre: DEBUG MARKER: == sanity test 852: mkdir using intent lock for striped directory ========================================================== 19:19:36 (1734376776)
[20223.575260] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity test_852 needs \>= 2 MDTs
[20223.915538] Lustre: DEBUG MARKER: SKIP: sanity test_852 needs >= 2 MDTs
[20224.367939] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 900: umount should not race with any mgc requeue thread ========================================================== 19:19:38 \(1734376778\)
[20224.723330] Lustre: DEBUG MARKER: == sanity test 900: umount should not race with any mgc requeue thread ========================================================== 19:19:38 (1734376778)
[20225.693623] Lustre: lustre-MDT0000-mdc-ffff8961ce4ec000: Connection to lustre-MDT0000 (at 10.240.38.186@tcp) was lost; in progress operations using this service will wait for recovery to complete
[20246.173825] LustreError: MGC10.240.38.186@tcp: Connection to MGS (at 10.240.38.186@tcp) was lost; in progress operations using this service will fail
[20246.180505] Lustre: Evicted from MGS (at 10.240.38.186@tcp) after server handle changed from 0x5d837553f580e271 to 0x5d837553f581e5e1
[20246.182876] Lustre: MGC10.240.38.186@tcp: Connection restored to 10.240.38.186@tcp (at 10.240.38.186@tcp)
[20246.184507] Lustre: Skipped 1 previous similar message
[20246.197929] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0 req@ffff8961d27f2a40 x1818616259120896/t68719519511(68719519511) o101->lustre-MDT0000-mdc-ffff8961ce4ec000@10.240.38.186@tcp:12/10 lens 576/608 e 0 to 0 dl 1734376816 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'md5sum.0' uid:0 gid:0
[20246.203984] LustreError: 429910:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 4 previous similar messages
[20246.951705] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20247.771721] LustreError: 734210:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 sleeping for 20000ms
[20248.115915] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[20248.568124] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[20248.832149] Lustre: DEBUG MARKER: lctl get_param -n at_max
[20249.099872] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[20249.495921] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[20251.561431] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[20251.561431] if [ $running -ne 0 ] ; then
[20251.561431] echo Stopping client $(hostname) /mnt/lustre opts:;
[20251.561431] lsof /mnt/lustre || need_kill=no;
[20251.561431] if [ x != x -a x$need_kill != xno ]; then
[20251.561431] pids=$(lsof -t /mnt/lustre | sort -u);
[20251.561431] if
[20267.859754] LustreError: 734210:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 awake
[20267.863615] LustreError: 734210:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 sleeping for 20000ms
[20287.935815] LustreError: 734210:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 awake
[20287.941961] LustreError: 734210:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 sleeping for 20000ms
[20287.973790] Lustre: Unmounted lustre-client
[20287.979084] Lustre: Skipped 1 previous similar message
[20308.011851] LustreError: 734210:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 awake
[20308.013840] LustreError: 734210:0:(mgc_request.c:616:do_requeue()) failed processing log: -5
[20308.203983] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[20308.203983] if [ $running -ne 0 ] ; then
[20308.203983] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[20308.203983] lsof /mnt/lustre2 || need_kill=no;
[20308.203983] if [ x != x -a x$need_kill != xno ]; then
[20308.203983] pids=$(lsof -t /mnt/lustre2 | sort -u);
[20308.203983]
[20312.110532] Autotest: Test running for 335 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[20346.815413] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1 || true
[20348.303296] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[20348.303296] lctl dl | grep ' ST ' || true
[20348.699925] Key type lgssc unregistered
[20348.772244] LNet: 781603:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[20348.796165] LNet: Removed LNI 10.240.44.153@tcp
[20349.029493] Key type .llcrypt unregistered
[20349.030405] Key type ._llcrypt unregistered
[20349.695523] Key type ._llcrypt registered
[20349.696499] Key type .llcrypt registered
[20349.950975] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[20350.908259] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[20351.027294] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[20351.028477] LNet: Accept all, port 7988
[20352.584030] Key type lgssc registered
[20353.204846] Lustre: Echo OBD driver; http://www.lustre.org/
[20357.402273] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[20357.402273] mpts=$(mount | grep -c /mnt/lustre' ');
[20357.402273] if [ $running -ne $mpts ]; then
[20357.402273] echo $(hostname) env are INSANE!;
[20357.402273] exit 1;
[20357.402273] fi
[20358.070425] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[20358.070425] mpts=$(mount | grep -c /mnt/lustre2' ');
[20358.070425] if [ $running -ne $mpts ]; then
[20358.070425] echo $(hostname) env are INSANE!;
[20358.070425] exit 1;
[20358.070425] fi
[20428.049373] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[20428.074806] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[20434.354562] Lustre: Mounted lustre-client
[20434.560436] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[20435.216480] Lustre: DEBUG MARKER:
[20435.216480] running=$(mount | grep -c /mnt/lustre' ');
[20435.216480] rc=0;
[20435.216480] if [ $running -eq 0 ] ; then
[20435.216480] mkdir -p /mnt/lustre;
[20435.216480] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[20435.216480] rc=$?;
[20435.216480] fi;
[20435.216480] exit $rc
[20435.953078] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[20440.663706] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[20441.015416] Lustre: DEBUG MARKER: Using TIMEOUT=20
[20441.427399] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[20442.274158] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[20449.434348] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 901: don\'t leak a mgc lock on client umount ========================================================== 19:23:23 \(1734377003\)
[20449.781215] Lustre: DEBUG MARKER: == sanity test 901: don't leak a mgc lock on client umount ========================================================== 19:23:23 (1734377003)
[20450.929215] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[20450.943353] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[20451.097960] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[20451.146570] Lustre: Unmounted lustre-client
[20451.158844] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[20451.173101] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[20451.243610] Lustre: Mounted lustre-client
[20453.006881] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 902: test short write doesn\'t hang lustre ========================================================== 19:23:27 \(1734377007\)
[20453.372910] Lustre: DEBUG MARKER: == sanity test 902: test short write doesn't hang lustre ========================================================== 19:23:27 (1734377007)
[20453.549654] Lustre: *** cfs_fail_loc=2001415, val=0***
[20455.654334] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 903: Test long page discard does not cause evictions ========================================================== 19:23:29 \(1734377009\)
[20456.014512] Lustre: DEBUG MARKER: == sanity test 903: Test long page discard does not cause evictions ========================================================== 19:23:29 (1734377009)
[20462.021959] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[20482.108313] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[20482.119143] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[20502.212366] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[20502.222672] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[20522.300410] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[20522.320069] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[20542.400460] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[20542.411629] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[20562.500524] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[20562.511661] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[20582.612582] LustreError: 785678:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[20591.557302] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 904: virtual project ID xattr ============= 19:25:45 \(1734377145\)
[20591.911562] Lustre: DEBUG MARKER: == sanity test 904: virtual project ID xattr ============= 19:25:45 (1734377145)
[20596.047418] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 905: bad or new opcode should not stuck client ========================================================== 19:25:50 \(1734377150\)
[20596.394128] Lustre: DEBUG MARKER: == sanity test 905: bad or new opcode should not stuck client ========================================================== 19:25:50 (1734377150)
[20596.986174] LustreError: lustre-OST0001-osc-ffff8961d57bc800: operation ost_ladvise to node 10.240.45.50@tcp failed: rc = -95
[20598.735797] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 906: Simple test for io_uring I\/O engine via fio ========================================================== 19:25:52 \(1734377152\)
[20599.086407] Lustre: DEBUG MARKER: == sanity test 906: Simple test for io_uring I/O engine via fio ========================================================== 19:25:52 (1734377152)
[20613.072013] Autotest: Test running for 340 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[20913.803520] Autotest: Test running for 345 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[20938.136035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 907: write rpc error during unlink ======== 19:31:32 \(1734377492\)
[20938.497969] Lustre: DEBUG MARKER: == sanity test 907: write rpc error during unlink ======== 19:31:32 (1734377492)
[20940.245748] LustreError: lustre-OST0000-osc-ffff8961d57bc800: operation ost_write to node 10.240.45.50@tcp failed: rc = -3
[20940.247783] LustreError: Skipped 13 previous similar messages
[20941.998173] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test complete, duration 20128 sec ============== 19:31:36 \(1734377496\)
[20942.368839] Lustre: DEBUG MARKER: == sanity test complete, duration 20128 sec ============== 19:31:36 (1734377496)
[20942.796432] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: start cleanup 19:31:36 \(1734377496\) ===
[20943.144630] Lustre: DEBUG MARKER: === sanity: start cleanup 19:31:36 (1734377496) ===
[20984.899270] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: finish cleanup 19:32:18 \(1734377538\) ===
[20985.244969] Lustre: DEBUG MARKER: === sanity: finish cleanup 19:32:18 (1734377538) ===
[20985.654241] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: sanity-lnet ============----- Mon Dec 16 19:32:19 UTC 2024
[20985.984877] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Mon Dec 16 19:32:19 UTC 2024
[20987.705518] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[20990.973943] Lustre: DEBUG MARKER: cat /etc/system-release
[20990.986290] Lustre: DEBUG MARKER: test -r /etc/os-release
[20990.997747] Lustre: DEBUG MARKER: cat /etc/os-release
[20991.244529] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[20991.244529] if [ $running -ne 0 ] ; then
[20991.244529] echo Stopping client $(hostname) /mnt/lustre opts:;
[20991.244529] lsof /mnt/lustre || need_kill=no;
[20991.244529] if [ x != x -a x$need_kill != xno ]; then
[20991.244529] pids=$(lsof -t /mnt/lustre | sort -u);
[20991.244529] if
[21014.464491] Lustre: Unmounted lustre-client
[21014.647162] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[21014.647162] if [ $running -ne 0 ] ; then
[21014.647162] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[21014.647162] lsof /mnt/lustre2 || need_kill=no;
[21014.647162] if [ x != x -a x$need_kill != xno ]; then
[21014.647162] pids=$(lsof -t /mnt/lustre2 | sort -u);
[21014.647162]
[21049.095420] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21049.095420] lctl dl | grep ' ST ' || true
[21049.475778] Key type lgssc unregistered
[21049.553989] LNet: 795716:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21050.574011] LNet: Removed LNI 10.240.44.153@tcp
[21050.782393] Key type .llcrypt unregistered
[21050.783243] Key type ._llcrypt unregistered
[21064.335976] Key type ._llcrypt registered
[21064.336928] Key type .llcrypt registered
[21064.411344] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21065.185298] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21065.189674] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21065.191233] LNet: Accept all, port 7988
[21065.194318] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show
[21066.471276] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests: 253 254
[21066.887614] Lustre: DEBUG MARKER: excepting tests: 253 254
[21067.022039] LNet: 796791:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21067.037949] LNet: Removed LNI 10.240.44.153@tcp
[21067.044988] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21067.044988] lctl dl | grep ' ST ' || true
[21067.169906] Key type .llcrypt unregistered
[21067.170730] Key type ._llcrypt unregistered
[21067.411901] Key type ._llcrypt registered
[21067.412758] Key type .llcrypt registered
[21067.705764] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 19:33:41 \(1734377621\)
[21068.028153] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 19:33:41 (1734377621)
[21068.203390] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21068.972879] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21068.986436] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21068.989622] LNet: 797067:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21068.993764] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21069.002275] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21069.005273] LNet: 797074:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21070.578573] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 1: Add peer with single nid \(tcp\) ==== 19:33:44 \(1734377624\)
[21070.904771] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 19:33:44 (1734377624)
[21071.038567] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21071.041817] LNet: 797524:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21071.045910] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21071.056860] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp
[21072.620407] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 2: Add peer with single nid \(o2ib\) === 19:33:46 \(1734377626\)
[21072.958133] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 19:33:46 (1734377626)
[21073.076507] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21073.079824] LNet: 797982:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21073.083893] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21073.095293] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 2.2.2.2@o2ib
[21074.661637] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 19:33:48 \(1734377628\)
[21075.006409] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 19:33:48 (1734377628)
[21075.125791] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21075.129109] LNet: 798439:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21075.133277] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21075.144755] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib
[21076.703464] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 4: Add peer with nidrange \(tcp\) ====== 19:33:50 \(1734377630\)
[21077.033181] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 19:33:50 (1734377630)
[21077.171351] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21077.177150] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21077.188548] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp
[21077.249582] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp
[21078.806667] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 5: Add peer with nidrange \(o2ib\) ===== 19:33:52 \(1734377632\)
[21079.138585] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 19:33:52 (1734377632)
[21079.269201] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21079.272443] LNet: 799359:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21079.274003] LNet: 799359:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 1 previous similar message
[21079.278182] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21079.289406] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib
[21079.350593] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib
[21079.409924] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib
[21080.984305] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 6: Add peer with multiple nidranges == 19:33:55 \(1734377635\)
[21081.315704] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 19:33:55 (1734377635)
[21081.447513] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21081.453278] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21081.464808] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi
[21083.039574] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 7: Various peer delete tests ========= 19:33:57 \(1734377637\)
[21083.396417] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 19:33:57 (1734377637)
[21083.520055] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21083.526005] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21083.535627] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@tcp
[21083.541183] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@tcp
[21083.598412] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@o2ib
[21083.603901] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@o2ib
[21083.660079] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp
[21083.666022] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@tcp
[21083.722324] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib
[21083.728266] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@o2ib
[21083.785064] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib
[21083.791303] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@tcp
[21083.848235] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@gni
[21083.853822] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@gni
[21083.910233] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni
[21083.915924] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@gni
[21083.972022] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@kfi
[21083.977373] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@kfi
[21084.033224] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi
[21084.038904] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@kfi
[21084.094844] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi
[21084.101180] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@gni
[21084.157814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp
[21084.164279] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp
[21085.740630] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 8: Delete single secondary nid from peer \(tcp\) ========================================================== 19:33:59 \(1734377639\)
[21086.053285] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 19:33:59 (1734377639)
[21086.183820] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21086.189487] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21086.200511] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp
[21086.206374] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp
[21087.774222] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 9: Delete all secondary nids from peer \(tcp\) ========================================================== 19:34:01 \(1734377641\)
[21088.108382] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 19:34:01 (1734377641)
[21088.250774] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21088.254341] LNet: 801282:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21088.255828] LNet: 801282:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 3 previous similar messages
[21088.260026] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21088.271979] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp
[21088.278449] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp
[21089.868803] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 10: Delete single secondary nid from peer \(o2ib\) ========================================================== 19:34:03 \(1734377643\)
[21090.181313] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 19:34:03 (1734377643)
[21090.318178] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21090.323921] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21090.335048] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp
[21090.341015] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp
[21091.921234] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 11: Delete all secondary nids from peer \(o2ib\) ========================================================== 19:34:05 \(1734377645\)
[21092.240194] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 19:34:05 (1734377645)
[21092.373052] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21092.378965] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21092.390272] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp
[21092.396522] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp
[21093.963249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 12: Delete a secondary nid from peer \(tcp and o2ib\) ========================================================== 19:34:08 \(1734377648\)
[21094.297325] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 19:34:08 (1734377648)
[21094.435759] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21094.441609] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21094.452928] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp
[21094.459537] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp
[21096.030593] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 13: Delete all secondary nids from peer \(tcp and o2ib\) ========================================================== 19:34:10 \(1734377650\)
[21096.345195] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 19:34:10 (1734377650)
[21096.477475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21096.483235] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21096.494673] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib
[21096.501773] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib
[21098.071209] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 14: import peer create\/delete with single nid ========================================================== 19:34:12 \(1734377652\)
[21098.397164] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 19:34:12 (1734377652)
[21098.529808] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21098.535728] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21098.549395] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21098.609641] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[21100.188624] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 15: import multi peer create\/delete with single nid per peer ========================================================== 19:34:14 \(1734377654\)
[21100.550625] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 19:34:14 (1734377654)
[21100.672545] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21100.678701] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21100.711828] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21100.749293] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[21102.338009] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 16: import peer create\/delete with multiple nids ========================================================== 19:34:16 \(1734377656\)
[21102.664056] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 19:34:16 (1734377656)
[21102.804142] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21102.809765] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21102.828617] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21102.864305] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[21104.442899] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 17: import multi peer create\/delete with multiple nids ========================================================== 19:34:18 \(1734377658\)
[21104.786494] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 19:34:18 (1734377658)
[21104.923555] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21104.927005] LNet: 805024:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21104.928542] LNet: 805024:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 7 previous similar messages
[21104.932699] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21105.001947] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21105.045408] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[21106.620291] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 19:34:20 \(1734377660\)
[21106.953146] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 19:34:20 (1734377660)
[21107.097750] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21107.103534] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21107.114955] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21107.123225] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[21108.720839] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 19:34:22 \(1734377662\)
[21109.059529] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 19:34:22 (1734377662)
[21109.200296] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21109.206226] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21109.217590] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21109.225914] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[21110.849738] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 19: Add peer with single nid \(gni\) === 19:34:24 \(1734377664\)
[21111.183582] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 19:34:24 (1734377664)
[21111.319373] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21111.325253] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21111.336455] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 19@gni
[21112.931160] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 19:34:27 \(1734377667\)
[21113.240265] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 19:34:27 (1734377667)
[21113.378572] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21113.384100] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21113.395183] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib
[21114.959036] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 21: Add peer with nidrange \(gni\) ===== 19:34:29 \(1734377669\)
[21115.293313] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 19:34:29 (1734377669)
[21115.433841] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21115.439529] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21115.451104] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni
[21115.512054] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni
[21117.070214] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 22: Delete single secondary nid from peer \(gni\) ========================================================== 19:34:31 \(1734377671\)
[21117.402410] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 19:34:31 (1734377671)
[21117.543735] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21117.549442] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21117.560616] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni
[21117.566286] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 22@gni --nid 26@gni
[21119.136547] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 23: Delete all secondary nids from peer \(gni\) ========================================================== 19:34:33 \(1734377673\)
[21119.467967] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 19:34:33 (1734377673)
[21119.607863] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21119.613708] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21119.624896] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni
[21119.630691] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni
[21121.186848] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 24: Delete a secondary nid from peer \(tcp, o2ib and gni\) ========================================================== 19:34:35 \(1734377675\)
[21121.522171] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 19:34:35 (1734377675)
[21121.673387] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21121.679178] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21121.690302] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni
[21121.696998] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp
[21123.241022] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 25: Delete all secondary nids from peer \(tcp, gni and o2ib\) ========================================================== 19:34:37 \(1734377677\)
[21123.572493] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 19:34:37 (1734377677)
[21123.711121] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21123.716678] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21123.727612] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni
[21123.734035] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni
[21125.275297] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 19:34:39 \(1734377679\)
[21125.613566] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 19:34:39 (1734377679)
[21125.748010] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21125.753657] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21125.761734] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim
[21125.772011] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@tcp
[21125.775244] LNetError: 809721:0:(peer.c:2184:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs
[21125.782604] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@tcp --force
[21125.787959] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show --nid 1.1.1.1@tcp
[21127.276077] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 19:34:41 \(1734377681\)
[21127.607246] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 19:34:41 (1734377681)
[21127.739259] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21127.745043] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21127.757058] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/tmp.RklA9hFUWc
[21127.767817] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/tmp.RklA9hFUWc
[21127.775674] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/tmp.RklA9hFUWc
[21129.267564] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 28: Test peer_list =================== 19:34:43 \(1734377683\)
[21129.599029] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 19:34:43 (1734377683)
[21129.737523] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21129.743228] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21129.751344] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp
[21129.756955] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp
[21129.763750] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer list
[21131.263352] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 19:34:45 \(1734377685\)
[21131.593440] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 19:34:45 (1734377685)
[21131.731136] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21131.736900] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21131.745061] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid foobar
[21131.750682] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid foobar
[21131.756176] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@o2ib
[21131.761765] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --nid 1.1.1.1@tcp
[21131.767277] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --nid 1.1.1.1@tcp
[21131.772737] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add
[21131.777906] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar
[21131.783870] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp
[21131.790105] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1@foo
[21131.795720] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1@tcp
[21131.801172] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1@o2ib
[21131.806703] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@gni
[21131.812584] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni
[21131.818385] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni
[21131.824161] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp
[21131.830199] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp
[21131.836249] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp
[21131.842429] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar
[21131.848280] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 1@gni
[21131.851757] LNetError: 811141:0:(router.c:630:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni
[21133.379642] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 19:34:47 \(1734377687\)
[21133.690558] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 19:34:47 (1734377687)
[21133.829542] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21133.835118] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21133.846272] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21135.394349] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 100: Add route with single gw ======== 19:34:49 \(1734377689\)
[21135.729477] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 19:34:49 (1734377689)
[21187.347998] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21187.694863] LNet: 812444:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21187.696474] LNet: 812444:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 14 previous similar messages
[21187.790484] Key type .llcrypt unregistered
[21187.791370] Key type ._llcrypt unregistered
[21189.825557] Key type ._llcrypt registered
[21189.826466] Key type .llcrypt registered
[21190.059210] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21190.461704] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21190.823768] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21192.452698] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21214.714606] Autotest: Test running for 350 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[21221.437851] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21221.441729] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21221.443124] LNet: Accept all, port 7988
[21222.798462] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.45.178@tcp
[21222.802050] LNet: 813427:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21222.805398] LNetError: 812905:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21222.808002] LNetError: 812905:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.45.178@tcp has gone from up to down
[21222.810136] LNetError: 813428:0:(router.c:333:lnet_is_route_alive()) peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21222.854454] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.45.178@tcp 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp1 via 10.240.45.178@tcp"; /usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.45.178@tcp; e
[21223.459061] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21223.782187] LNet: 813553:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21223.798365] LNet: Removed LNI 10.240.44.153@tcp
[21223.882415] Key type .llcrypt unregistered
[21223.883305] Key type ._llcrypt unregistered
[21225.391261] Key type ._llcrypt registered
[21225.392119] Key type .llcrypt registered
[21225.634706] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 101: Add route with multiple gw ====== 19:36:19 \(1734377779\)
[21225.966690] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 19:36:19 (1734377779)
[21226.144417] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21226.913067] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21226.921640] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21226.925297] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21226.926560] LNet: Accept all, port 7988
[21227.036035] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.44.[154-158]@tcp
[21227.040194] LNet: 814197:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21227.043400] LNetError: 814072:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.44.156@tcp is being used as a gateway but routing feature is not turned on
[21227.045854] LNetError: 814072:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.44.156@tcp has gone from up to down
[21227.048429] LNetError: 814199:0:(router.c:333:lnet_is_route_alive()) peer 10.240.44.156@tcp is being used as a gateway but routing feature is not turned on
[21228.638834] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 102: Delete route with single gw ===== 19:36:22 \(1734377782\)
[21228.970686] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 19:36:22 (1734377782)
[21229.099678] LNetError: 814663:0:(router.c:333:lnet_is_route_alive()) peer 10.240.44.157@tcp is being used as a gateway but routing feature is not turned on
[21229.102027] LNetError: 814663:0:(router.c:333:lnet_is_route_alive()) Skipped 2 previous similar messages
[21238.763838] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21239.115257] LNet: 815067:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21240.142429] LNet: Removed LNI 10.240.44.153@tcp
[21240.238311] Key type .llcrypt unregistered
[21240.239196] Key type ._llcrypt unregistered
[21242.161308] Key type ._llcrypt registered
[21242.162152] Key type .llcrypt registered
[21242.393674] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21242.837851] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21243.163718] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21244.771586] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21251.461561] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21251.465281] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21251.466724] LNet: Accept all, port 7988
[21252.515104] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.45.178@tcp
[21252.519040] LNet: 815779:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21252.522158] LNetError: 815529:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21252.523587] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.45.178@tcp
[21252.524619] LNetError: 815529:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.45.178@tcp has gone from up to down
[21252.571953] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.45.178@tcp 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp1 via 10.240.45.178@tcp"; /usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.45.178@tcp; e
[21253.158856] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21253.489769] LNet: 815905:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21253.506348] LNet: Removed LNI 10.240.44.153@tcp
[21253.594342] Key type .llcrypt unregistered
[21253.595191] Key type ._llcrypt unregistered
[21255.070195] Key type ._llcrypt registered
[21255.071090] Key type .llcrypt registered
[21255.313434] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 103: Delete route with multiple gw === 19:36:49 \(1734377809\)
[21255.642323] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 19:36:49 (1734377809)
[21255.818784] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21256.584734] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21256.593537] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21256.597063] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21256.598168] LNet: Accept all, port 7988
[21256.683600] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp103 --gateway 10.240.44.[154-159/2]@tcp
[21256.687426] LNet: 816524:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21256.690028] LNetError: 816421:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp103 through 10.240.44.158@tcp has gone from up to down
[21256.692592] LNetError: 816415:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.44.154@tcp is being used as a gateway but routing feature is not turned on
[21256.697511] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route del --net tcp103 --gateway 10.240.44.[154-159/2]@tcp
[21258.286113] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 104: Set\/check response_tracking param ========================================================== 19:36:52 \(1734377812\)
[21258.616642] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 19:36:52 (1734377812)
[21258.748336] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21258.751819] LNet: 816989:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21259.790484] LNet: Removed LNI 10.240.44.153@tcp
[21259.794670] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.808030] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking -1
[21259.817995] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21259.821190] LNet: 817003:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21259.825110] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.834871] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21259.844351] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21259.849940] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.858113] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 0
[21259.870775] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21259.876406] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.886137] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21259.898937] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21259.904483] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.912747] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 1
[21259.925521] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21259.931138] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.940879] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21259.953623] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21259.959316] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.967581] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 2
[21259.980386] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21259.986071] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21259.995770] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21260.008190] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21260.013672] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21260.021732] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 3
[21260.034436] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21260.039811] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21260.049441] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21260.061915] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21260.067407] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21260.075472] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 4
[21260.085223] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21260.090637] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21260.100286] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[21261.610648] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 19:36:55 \(1734377815\)
[21261.939673] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 19:36:55 (1734377815)
[21271.750771] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21272.089105] LNet: 817987:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21272.090710] LNet: 817987:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 10 previous similar messages
[21272.182426] Key type .llcrypt unregistered
[21272.183238] Key type ._llcrypt unregistered
[21274.218981] Key type ._llcrypt registered
[21274.219873] Key type .llcrypt registered
[21274.450837] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21274.868908] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21275.224417] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21276.825245] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21283.478939] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21283.482818] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21283.483919] LNet: Accept all, port 7988
[21284.538247] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.45.178@tcp
[21284.542040] LNet: 818696:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21284.545314] LNetError: 818448:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21284.547834] LNetError: 818448:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.45.178@tcp has gone from up to down
[21284.549154] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim 10.240.45.178@tcp
[21284.553327] LNetError: 818698:0:(peer.c:374:lnet_peer_ni_del_locked()) Peer NI 10.240.45.178@tcp is a gateway. Can not delete it
[21284.555299] LNetError: 818698:0:(peer.c:2176:lnet_del_peer_ni()) 10.240.45.178@tcp is a router. Can not be deleted
[21284.566453] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.45.178@tcp 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp1 via 10.240.45.178@tcp"; /usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.45.178@tcp; e
[21284.577392] LNetError: 818711:0:(router.c:333:lnet_is_route_alive()) peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21285.154861] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21285.520147] LNet: 818804:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21285.534494] LNet: Removed LNI 10.240.44.153@tcp
[21285.634522] Key type .llcrypt unregistered
[21285.635365] Key type ._llcrypt unregistered
[21287.074962] Key type ._llcrypt registered
[21287.075811] Key type .llcrypt registered
[21287.312743] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 106: Deleting GW peer should fail ==== 19:37:21 \(1734377841\)
[21287.641505] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 19:37:21 (1734377841)
[21297.516051] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21297.918436] Key type .llcrypt unregistered
[21297.919279] Key type ._llcrypt unregistered
[21299.987506] Key type ._llcrypt registered
[21299.988392] Key type .llcrypt registered
[21300.220071] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21300.696116] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21301.125495] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21302.614068] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21309.298780] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21309.302818] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21309.303904] LNet: Accept all, port 7988
[21310.349323] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.45.178@tcp
[21310.353059] LNet: 820381:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21310.356239] LNetError: 820133:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21310.358714] LNetError: 820133:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.45.178@tcp has gone from up to down
[21310.360139] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim 10.240.45.178@tcp
[21310.364302] LNetError: 820383:0:(peer.c:2176:lnet_del_peer_ni()) 10.240.45.178@tcp is a router. Can not be deleted
[21310.375321] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.45.178@tcp 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp1 via 10.240.45.178@tcp"; /usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.45.178@tcp; e
[21310.386032] LNetError: 820396:0:(router.c:333:lnet_is_route_alive()) peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21310.973738] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21311.312989] LNet: 820489:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21312.334525] LNet: Removed LNI 10.240.44.153@tcp
[21312.418571] Key type .llcrypt unregistered
[21312.419405] Key type ._llcrypt unregistered
[21313.762070] Key type ._llcrypt registered
[21313.762929] Key type .llcrypt registered
[21314.000711] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 107: Deleting extra interface doesn\'t crash node ========================================================== 19:37:48 \(1734377868\)
[21314.329505] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 19:37:48 (1734377868)
[21314.469726] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21314.469726] lctl dl | grep ' ST ' || true
[21314.526499] Key type .llcrypt unregistered
[21314.527309] Key type ._llcrypt unregistered
[21314.758847] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21314.760162] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21314.826150] Key type ._llcrypt registered
[21314.827052] Key type .llcrypt registered
[21314.882120] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21315.663919] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21315.672697] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21315.676183] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21315.677242] LNet: Accept all, port 7988
[21315.759008] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[21315.762260] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21316.002189] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp --if test1pg
[21316.005586] LNet: Removed LNI 10.1.2.3@tcp
[21316.046243] LNet: 821634:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21316.066615] LNet: Removed LNI 10.240.44.153@tcp
[21316.074313] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21316.074313] lctl dl | grep ' ST ' || true
[21316.214552] Key type .llcrypt unregistered
[21316.215373] Key type ._llcrypt unregistered
[21317.732338] Key type ._llcrypt registered
[21317.733205] Key type .llcrypt registered
[21317.969389] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 108: Check Multi-Rail setup ========== 19:37:52 \(1734377872\)
[21318.301180] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 19:37:52 (1734377872)
[21318.441671] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21318.441671] lctl dl | grep ' ST ' || true
[21318.502484] Key type .llcrypt unregistered
[21318.503281] Key type ._llcrypt unregistered
[21318.746984] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21318.748250] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21318.815028] Key type ._llcrypt registered
[21318.817459] Key type .llcrypt registered
[21318.890758] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21319.660384] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21319.676122] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21319.679879] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21319.681011] LNet: Accept all, port 7988
[21319.770523] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[21319.773766] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21320.110917] LNet: 822872:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21320.112622] LNet: Removed LNI 10.240.44.153@tcp
[21320.126944] LNet: Removed LNI 10.1.2.3@tcp
[21320.137137] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21320.137137] lctl dl | grep ' ST ' || true
[21320.318575] Key type .llcrypt unregistered
[21320.319426] Key type ._llcrypt unregistered
[21321.836251] Key type ._llcrypt registered
[21321.837102] Key type .llcrypt registered
[21322.074849] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 109: Add NI using a network interface alias \(LU-16859\) ========================================================== 19:37:56 \(1734377876\)
[21322.409418] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 19:37:56 (1734377876)
[21322.544249] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21322.544249] lctl dl | grep ' ST ' || true
[21322.602586] Key type .llcrypt unregistered
[21322.603383] Key type ._llcrypt unregistered
[21322.775678] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21322.776934] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21322.856149] Key type ._llcrypt registered
[21322.859591] Key type .llcrypt registered
[21322.934353] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21323.721272] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21323.729992] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg:0
[21323.733719] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180]
[21323.734953] LNet: Accept all, port 7988
[21323.972275] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[21323.975698] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21324.228113] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp --if test1pg
[21324.231545] LNet: Removed LNI 10.1.2.3@tcp
[21324.234354] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp --if test1pg:0
[21324.246562] LNet: Removed LNI 10.1.2.31@tcp
[21324.249412] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[21324.253070] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21324.343454] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg:0
[21324.347240] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180]
[21324.482064] LNet: 824514:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21324.483761] LNet: Removed LNI 10.1.2.3@tcp
[21324.498645] LNet: Removed LNI 10.1.2.31@tcp
[21324.509116] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21324.509116] lctl dl | grep ' ST ' || true
[21324.706592] Key type .llcrypt unregistered
[21324.707396] Key type ._llcrypt unregistered
[21326.255601] Key type ._llcrypt registered
[21326.256492] Key type .llcrypt registered
[21326.494043] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 19:38:00 \(1734377880\)
[21326.823859] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 19:38:00 (1734377880)
[21327.009277] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21327.777719] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21327.790955] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21327.794682] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21327.795771] LNet: Accept all, port 7988
[21327.893990] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21327.897237] LNet: 825280:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21327.914542] LNet: Removed LNI 10.240.44.153@tcp
[21327.917910] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21327.926074] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --nid 10.240.44.153@tcp
[21327.929677] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21327.930893] LNet: Accept all, port 7988
[21328.022635] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --nid 10.240.44.153@tcp
[21328.038596] LNet: Removed LNI 10.240.44.153@tcp
[21328.054931] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21328.054931] lctl dl | grep ' ST ' || true
[21328.174529] Key type .llcrypt unregistered
[21328.175328] Key type ._llcrypt unregistered
[21329.699875] Key type ._llcrypt registered
[21329.700738] Key type .llcrypt registered
[21329.937148] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 111: Test many routes ================ 19:38:04 \(1734377884\)
[21330.273999] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 19:38:04 (1734377884)
[21340.086209] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21340.462553] Key type .llcrypt unregistered
[21340.463409] Key type ._llcrypt unregistered
[21342.565292] Key type ._llcrypt registered
[21342.566144] Key type .llcrypt registered
[21342.796851] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21343.199186] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21343.550734] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21345.187697] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21351.832457] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21351.836330] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21351.837424] LNet: Accept all, port 7988
[21352.908932] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp12 -g 10.240.45.178@tcp
[21352.912660] LNet: 827078:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21352.915860] LNetError: 826817:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21352.917449] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp13 -g 10.240.45.178@tcp
[21352.918302] LNetError: 826817:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp12 through 10.240.45.178@tcp has gone from up to down
[21352.926008] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp14 -g 10.240.45.178@tcp
[21352.931693] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp15 -g 10.240.45.178@tcp
[21352.937365] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp16 -g 10.240.45.178@tcp
[21352.943110] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp17 -g 10.240.45.178@tcp
[21352.948982] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp18 -g 10.240.45.178@tcp
[21352.954730] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp19 -g 10.240.45.178@tcp
[21352.960500] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp110 -g 10.240.45.178@tcp
[21352.966094] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp111 -g 10.240.45.178@tcp
[21352.971633] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp112 -g 10.240.45.178@tcp
[21352.977301] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp113 -g 10.240.45.178@tcp
[21352.983017] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp114 -g 10.240.45.178@tcp
[21352.988729] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp115 -g 10.240.45.178@tcp
[21352.994361] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp116 -g 10.240.45.178@tcp
[21353.000107] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp117 -g 10.240.45.178@tcp
[21353.005816] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp118 -g 10.240.45.178@tcp
[21353.011534] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp119 -g 10.240.45.178@tcp
[21353.017166] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp120 -g 10.240.45.178@tcp
[21353.022919] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp121 -g 10.240.45.178@tcp
[21353.028561] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp122 -g 10.240.45.178@tcp
[21353.034154] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp123 -g 10.240.45.178@tcp
[21353.039713] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp124 -g 10.240.45.178@tcp
[21353.045339] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp125 -g 10.240.45.178@tcp
[21353.051061] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp126 -g 10.240.45.178@tcp
[21353.056764] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp127 -g 10.240.45.178@tcp
[21353.062325] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp128 -g 10.240.45.178@tcp
[21353.067953] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp129 -g 10.240.45.178@tcp
[21353.073644] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp130 -g 10.240.45.178@tcp
[21353.079195] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp131 -g 10.240.45.178@tcp
[21353.084862] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp132 -g 10.240.45.178@tcp
[21353.090441] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp133 -g 10.240.45.178@tcp
[21353.096144] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp134 -g 10.240.45.178@tcp
[21353.101835] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp135 -g 10.240.45.178@tcp
[21353.107423] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp136 -g 10.240.45.178@tcp
[21353.113029] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp137 -g 10.240.45.178@tcp
[21353.118813] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp138 -g 10.240.45.178@tcp
[21353.124366] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp139 -g 10.240.45.178@tcp
[21353.129971] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp140 -g 10.240.45.178@tcp
[21353.135693] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp141 -g 10.240.45.178@tcp
[21353.141372] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp142 -g 10.240.45.178@tcp
[21353.147143] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp143 -g 10.240.45.178@tcp
[21353.152644] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp144 -g 10.240.45.178@tcp
[21353.158260] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp145 -g 10.240.45.178@tcp
[21353.163947] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp146 -g 10.240.45.178@tcp
[21353.169526] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp147 -g 10.240.45.178@tcp
[21353.175249] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp148 -g 10.240.45.178@tcp
[21353.180832] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp149 -g 10.240.45.178@tcp
[21353.186483] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp150 -g 10.240.45.178@tcp
[21353.192152] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp151 -g 10.240.45.178@tcp
[21353.197714] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp152 -g 10.240.45.178@tcp
[21353.203432] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp153 -g 10.240.45.178@tcp
[21353.208974] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp154 -g 10.240.45.178@tcp
[21353.214681] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp155 -g 10.240.45.178@tcp
[21353.220244] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp156 -g 10.240.45.178@tcp
[21353.225960] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp157 -g 10.240.45.178@tcp
[21353.231689] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp158 -g 10.240.45.178@tcp
[21353.237374] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp159 -g 10.240.45.178@tcp
[21353.243151] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp160 -g 10.240.45.178@tcp
[21353.248795] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp161 -g 10.240.45.178@tcp
[21353.254330] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp162 -g 10.240.45.178@tcp
[21353.260053] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp163 -g 10.240.45.178@tcp
[21353.265725] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp164 -g 10.240.45.178@tcp
[21353.271476] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp165 -g 10.240.45.178@tcp
[21353.277152] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp166 -g 10.240.45.178@tcp
[21353.282865] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp167 -g 10.240.45.178@tcp
[21353.288537] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp168 -g 10.240.45.178@tcp
[21353.294287] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp169 -g 10.240.45.178@tcp
[21353.300057] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp170 -g 10.240.45.178@tcp
[21353.305747] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp171 -g 10.240.45.178@tcp
[21353.311486] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp172 -g 10.240.45.178@tcp
[21353.317156] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp173 -g 10.240.45.178@tcp
[21353.322952] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp174 -g 10.240.45.178@tcp
[21353.328559] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp175 -g 10.240.45.178@tcp
[21353.334186] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp176 -g 10.240.45.178@tcp
[21353.339870] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp177 -g 10.240.45.178@tcp
[21353.345568] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp178 -g 10.240.45.178@tcp
[21353.351206] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp179 -g 10.240.45.178@tcp
[21353.356882] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp180 -g 10.240.45.178@tcp
[21353.362498] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp181 -g 10.240.45.178@tcp
[21353.368261] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp182 -g 10.240.45.178@tcp
[21353.373911] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp183 -g 10.240.45.178@tcp
[21353.379616] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp184 -g 10.240.45.178@tcp
[21353.385242] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp185 -g 10.240.45.178@tcp
[21353.390884] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp186 -g 10.240.45.178@tcp
[21353.396655] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp187 -g 10.240.45.178@tcp
[21353.402293] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp188 -g 10.240.45.178@tcp
[21353.407972] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp189 -g 10.240.45.178@tcp
[21353.411523] LNet: 827252:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21353.413625] LNet: 827252:0:(router.c:737:lnet_add_route()) Skipped 86 previous similar messages
[21353.417322] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp190 -g 10.240.45.178@tcp
[21353.421363] LNetError: 826817:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21353.423840] LNetError: 826817:0:(router.c:399:lnet_router_discovery_ping_reply()) Skipped 88 previous similar messages
[21353.425423] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp191 -g 10.240.45.178@tcp
[21353.425615] LNetError: 826817:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp190 through 10.240.45.178@tcp has gone from up to down
[21353.429142] LNetError: 826817:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) Skipped 87 previous similar messages
[21353.432866] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp192 -g 10.240.45.178@tcp
[21353.438649] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp193 -g 10.240.45.178@tcp
[21353.444318] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp194 -g 10.240.45.178@tcp
[21353.450052] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp195 -g 10.240.45.178@tcp
[21353.455911] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp196 -g 10.240.45.178@tcp
[21353.461589] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp197 -g 10.240.45.178@tcp
[21353.467354] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp198 -g 10.240.45.178@tcp
[21353.473062] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp199 -g 10.240.45.178@tcp
[21353.478902] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1100 -g 10.240.45.178@tcp
[21353.484709] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1101 -g 10.240.45.178@tcp
[21353.490321] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1102 -g 10.240.45.178@tcp
[21353.496041] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1103 -g 10.240.45.178@tcp
[21353.501652] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1104 -g 10.240.45.178@tcp
[21353.507342] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1105 -g 10.240.45.178@tcp
[21353.513001] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1106 -g 10.240.45.178@tcp
[21353.518728] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1107 -g 10.240.45.178@tcp
[21353.524393] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1108 -g 10.240.45.178@tcp
[21353.530088] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1109 -g 10.240.45.178@tcp
[21353.535878] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1110 -g 10.240.45.178@tcp
[21353.541484] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1111 -g 10.240.45.178@tcp
[21353.547161] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1112 -g 10.240.45.178@tcp
[21353.552880] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1113 -g 10.240.45.178@tcp
[21353.558452] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1114 -g 10.240.45.178@tcp
[21353.564105] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1115 -g 10.240.45.178@tcp
[21353.569767] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1116 -g 10.240.45.178@tcp
[21353.575568] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1117 -g 10.240.45.178@tcp
[21353.581270] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1118 -g 10.240.45.178@tcp
[21353.587033] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1119 -g 10.240.45.178@tcp
[21353.592763] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1120 -g 10.240.45.178@tcp
[21353.598345] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1121 -g 10.240.45.178@tcp
[21353.604126] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1122 -g 10.240.45.178@tcp
[21353.609877] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1123 -g 10.240.45.178@tcp
[21353.615640] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1124 -g 10.240.45.178@tcp
[21353.621338] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1125 -g 10.240.45.178@tcp
[21353.627087] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1126 -g 10.240.45.178@tcp
[21353.632864] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1127 -g 10.240.45.178@tcp
[21353.638475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1128 -g 10.240.45.178@tcp
[21353.644311] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1129 -g 10.240.45.178@tcp
[21353.650039] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1130 -g 10.240.45.178@tcp
[21353.655755] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1131 -g 10.240.45.178@tcp
[21353.661617] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1132 -g 10.240.45.178@tcp
[21353.667344] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1133 -g 10.240.45.178@tcp
[21353.673167] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1134 -g 10.240.45.178@tcp
[21353.678974] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1135 -g 10.240.45.178@tcp
[21353.684675] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1136 -g 10.240.45.178@tcp
[21353.690480] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1137 -g 10.240.45.178@tcp
[21353.696270] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1138 -g 10.240.45.178@tcp
[21353.702001] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1139 -g 10.240.45.178@tcp
[21353.707814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1140 -g 10.240.45.178@tcp
[21353.713584] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1141 -g 10.240.45.178@tcp
[21353.719322] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1142 -g 10.240.45.178@tcp
[21353.725062] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1143 -g 10.240.45.178@tcp
[21353.730839] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1144 -g 10.240.45.178@tcp
[21353.736465] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1145 -g 10.240.45.178@tcp
[21353.742206] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1146 -g 10.240.45.178@tcp
[21353.748114] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1147 -g 10.240.45.178@tcp
[21353.753733] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1148 -g 10.240.45.178@tcp
[21353.759440] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1149 -g 10.240.45.178@tcp
[21353.765162] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1150 -g 10.240.45.178@tcp
[21353.771127] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1151 -g 10.240.45.178@tcp
[21353.776801] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1152 -g 10.240.45.178@tcp
[21353.782502] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1153 -g 10.240.45.178@tcp
[21353.788317] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1154 -g 10.240.45.178@tcp
[21353.794101] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1155 -g 10.240.45.178@tcp
[21353.799686] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1156 -g 10.240.45.178@tcp
[21353.805322] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1157 -g 10.240.45.178@tcp
[21353.811040] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1158 -g 10.240.45.178@tcp
[21353.816700] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1159 -g 10.240.45.178@tcp
[21353.822403] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1160 -g 10.240.45.178@tcp
[21353.828143] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1161 -g 10.240.45.178@tcp
[21353.833935] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1162 -g 10.240.45.178@tcp
[21353.839618] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1163 -g 10.240.45.178@tcp
[21353.845294] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1164 -g 10.240.45.178@tcp
[21353.851084] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1165 -g 10.240.45.178@tcp
[21353.856930] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1166 -g 10.240.45.178@tcp
[21353.862717] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1167 -g 10.240.45.178@tcp
[21353.868442] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1168 -g 10.240.45.178@tcp
[21353.874192] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1169 -g 10.240.45.178@tcp
[21353.879834] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1170 -g 10.240.45.178@tcp
[21353.885528] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1171 -g 10.240.45.178@tcp
[21353.891364] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1172 -g 10.240.45.178@tcp
[21353.897075] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1173 -g 10.240.45.178@tcp
[21353.902737] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1174 -g 10.240.45.178@tcp
[21353.908426] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1175 -g 10.240.45.178@tcp
[21353.914116] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1176 -g 10.240.45.178@tcp
[21353.919917] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1177 -g 10.240.45.178@tcp
[21353.925655] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1178 -g 10.240.45.178@tcp
[21353.931402] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1179 -g 10.240.45.178@tcp
[21353.937142] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1180 -g 10.240.45.178@tcp
[21353.942928] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1181 -g 10.240.45.178@tcp
[21353.948631] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1182 -g 10.240.45.178@tcp
[21353.954490] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1183 -g 10.240.45.178@tcp
[21353.960332] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1184 -g 10.240.45.178@tcp
[21353.966033] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1185 -g 10.240.45.178@tcp
[21353.971665] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1186 -g 10.240.45.178@tcp
[21353.977408] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1187 -g 10.240.45.178@tcp
[21353.983005] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1188 -g 10.240.45.178@tcp
[21353.988689] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1189 -g 10.240.45.178@tcp
[21353.994349] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1190 -g 10.240.45.178@tcp
[21354.000074] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1191 -g 10.240.45.178@tcp
[21354.005870] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1192 -g 10.240.45.178@tcp
[21354.011583] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1193 -g 10.240.45.178@tcp
[21354.017417] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1194 -g 10.240.45.178@tcp
[21354.023043] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1195 -g 10.240.45.178@tcp
[21354.028769] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1196 -g 10.240.45.178@tcp
[21354.034649] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1197 -g 10.240.45.178@tcp
[21354.040300] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1198 -g 10.240.45.178@tcp
[21354.046032] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1199 -g 10.240.45.178@tcp
[21354.051798] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1200 -g 10.240.45.178@tcp
[21354.057595] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1201 -g 10.240.45.178@tcp
[21354.063328] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1202 -g 10.240.45.178@tcp
[21354.069038] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1203 -g 10.240.45.178@tcp
[21354.074885] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1204 -g 10.240.45.178@tcp
[21354.080592] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1205 -g 10.240.45.178@tcp
[21354.086271] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1206 -g 10.240.45.178@tcp
[21354.092131] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1207 -g 10.240.45.178@tcp
[21354.097694] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1208 -g 10.240.45.178@tcp
[21354.103572] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1209 -g 10.240.45.178@tcp
[21354.109261] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1210 -g 10.240.45.178@tcp
[21354.114943] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1211 -g 10.240.45.178@tcp
[21354.120620] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1212 -g 10.240.45.178@tcp
[21354.126321] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1213 -g 10.240.45.178@tcp
[21354.132052] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1214 -g 10.240.45.178@tcp
[21354.137947] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1215 -g 10.240.45.178@tcp
[21354.143699] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1216 -g 10.240.45.178@tcp
[21354.149373] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1217 -g 10.240.45.178@tcp
[21354.155270] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1218 -g 10.240.45.178@tcp
[21354.160967] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1219 -g 10.240.45.178@tcp
[21354.166797] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1220 -g 10.240.45.178@tcp
[21354.172476] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1221 -g 10.240.45.178@tcp
[21354.178274] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1222 -g 10.240.45.178@tcp
[21354.183867] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1223 -g 10.240.45.178@tcp
[21354.189562] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1224 -g 10.240.45.178@tcp
[21354.195508] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1225 -g 10.240.45.178@tcp
[21354.201191] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1226 -g 10.240.45.178@tcp
[21354.206772] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1227 -g 10.240.45.178@tcp
[21354.212475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1228 -g 10.240.45.178@tcp
[21354.218237] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1229 -g 10.240.45.178@tcp
[21354.224091] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1230 -g 10.240.45.178@tcp
[21354.229764] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1231 -g 10.240.45.178@tcp
[21354.235442] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1232 -g 10.240.45.178@tcp
[21354.241091] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1233 -g 10.240.45.178@tcp
[21354.246959] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1234 -g 10.240.45.178@tcp
[21354.252517] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1235 -g 10.240.45.178@tcp
[21354.258344] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1236 -g 10.240.45.178@tcp
[21354.264079] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1237 -g 10.240.45.178@tcp
[21354.269791] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1238 -g 10.240.45.178@tcp
[21354.275640] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1239 -g 10.240.45.178@tcp
[21354.281430] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1240 -g 10.240.45.178@tcp
[21354.287186] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1241 -g 10.240.45.178@tcp
[21354.292870] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1242 -g 10.240.45.178@tcp
[21354.298615] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1243 -g 10.240.45.178@tcp
[21354.304218] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1244 -g 10.240.45.178@tcp
[21354.309986] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1245 -g 10.240.45.178@tcp
[21354.315668] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1246 -g 10.240.45.178@tcp
[21354.321371] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1247 -g 10.240.45.178@tcp
[21354.327025] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1248 -g 10.240.45.178@tcp
[21354.332671] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1249 -g 10.240.45.178@tcp
[21354.338368] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1250 -g 10.240.45.178@tcp
[21354.344045] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1251 -g 10.240.45.178@tcp
[21354.349758] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1252 -g 10.240.45.178@tcp
[21354.355498] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1253 -g 10.240.45.178@tcp
[21354.361325] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1254 -g 10.240.45.178@tcp
[21354.366904] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1255 -g 10.240.45.178@tcp
[21354.372572] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1256 -g 10.240.45.178@tcp
[21354.378074] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1257 -g 10.240.45.178@tcp
[21354.383700] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1258 -g 10.240.45.178@tcp
[21354.389380] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1259 -g 10.240.45.178@tcp
[21354.394916] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1260 -g 10.240.45.178@tcp
[21354.400537] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1261 -g 10.240.45.178@tcp
[21354.406219] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1262 -g 10.240.45.178@tcp
[21354.411947] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1263 -g 10.240.45.178@tcp
[21354.415559] LNet: 827600:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21354.417694] LNet: 827600:0:(router.c:737:lnet_add_route()) Skipped 173 previous similar messages
[21354.419517] LNetError: 826817:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21354.421369] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1264 -g 10.240.45.178@tcp
[21354.421980] LNetError: 826817:0:(router.c:399:lnet_router_discovery_ping_reply()) Skipped 171 previous similar messages
[21354.427558] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1265 -g 10.240.45.178@tcp
[21354.431363] LNetError: 826817:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1265 through 10.240.45.178@tcp has gone from up to down
[21354.433512] LNetError: 826817:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) Skipped 173 previous similar messages
[21354.436447] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1266 -g 10.240.45.178@tcp
[21354.442092] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1267 -g 10.240.45.178@tcp
[21354.447860] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1268 -g 10.240.45.178@tcp
[21354.453569] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1269 -g 10.240.45.178@tcp
[21354.459305] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1270 -g 10.240.45.178@tcp
[21354.464996] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1271 -g 10.240.45.178@tcp
[21354.470801] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1272 -g 10.240.45.178@tcp
[21354.476486] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1273 -g 10.240.45.178@tcp
[21354.482158] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1274 -g 10.240.45.178@tcp
[21354.487832] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1275 -g 10.240.45.178@tcp
[21354.493602] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1276 -g 10.240.45.178@tcp
[21354.499358] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1277 -g 10.240.45.178@tcp
[21354.505050] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1278 -g 10.240.45.178@tcp
[21354.510868] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1279 -g 10.240.45.178@tcp
[21354.516556] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1280 -g 10.240.45.178@tcp
[21354.522250] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1281 -g 10.240.45.178@tcp
[21354.527961] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1282 -g 10.240.45.178@tcp
[21354.533711] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1283 -g 10.240.45.178@tcp
[21354.539566] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1284 -g 10.240.45.178@tcp
[21354.545245] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1285 -g 10.240.45.178@tcp
[21354.550958] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1286 -g 10.240.45.178@tcp
[21354.556726] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1287 -g 10.240.45.178@tcp
[21354.562355] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1288 -g 10.240.45.178@tcp
[21354.568143] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1289 -g 10.240.45.178@tcp
[21354.573875] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1290 -g 10.240.45.178@tcp
[21354.579641] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1291 -g 10.240.45.178@tcp
[21354.585295] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1292 -g 10.240.45.178@tcp
[21354.590936] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1293 -g 10.240.45.178@tcp
[21354.596545] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1294 -g 10.240.45.178@tcp
[21354.602197] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1295 -g 10.240.45.178@tcp
[21354.607926] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1296 -g 10.240.45.178@tcp
[21354.613638] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1297 -g 10.240.45.178@tcp
[21354.619475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1298 -g 10.240.45.178@tcp
[21354.625124] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1299 -g 10.240.45.178@tcp
[21354.630869] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1300 -g 10.240.45.178@tcp
[21354.636443] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1301 -g 10.240.45.178@tcp
[21354.642193] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1302 -g 10.240.45.178@tcp
[21354.647801] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1303 -g 10.240.45.178@tcp
[21354.653557] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1304 -g 10.240.45.178@tcp
[21354.659172] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1305 -g 10.240.45.178@tcp
[21354.664849] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1306 -g 10.240.45.178@tcp
[21354.670481] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1307 -g 10.240.45.178@tcp
[21354.676198] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1308 -g 10.240.45.178@tcp
[21354.681731] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1309 -g 10.240.45.178@tcp
[21354.687419] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1310 -g 10.240.45.178@tcp
[21354.693033] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1311 -g 10.240.45.178@tcp
[21354.698739] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1312 -g 10.240.45.178@tcp
[21354.704415] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1313 -g 10.240.45.178@tcp
[21354.709985] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1314 -g 10.240.45.178@tcp
[21354.715653] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1315 -g 10.240.45.178@tcp
[21354.721372] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1316 -g 10.240.45.178@tcp
[21354.727122] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1317 -g 10.240.45.178@tcp
[21354.732709] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1318 -g 10.240.45.178@tcp
[21354.738322] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1319 -g 10.240.45.178@tcp
[21354.744122] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1320 -g 10.240.45.178@tcp
[21354.750119] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1321 -g 10.240.45.178@tcp
[21354.755991] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1322 -g 10.240.45.178@tcp
[21354.761731] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1323 -g 10.240.45.178@tcp
[21354.767407] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1324 -g 10.240.45.178@tcp
[21354.773150] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1325 -g 10.240.45.178@tcp
[21354.778940] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1326 -g 10.240.45.178@tcp
[21354.784685] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1327 -g 10.240.45.178@tcp
[21354.790343] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1328 -g 10.240.45.178@tcp
[21354.796179] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1329 -g 10.240.45.178@tcp
[21354.801956] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1330 -g 10.240.45.178@tcp
[21354.807753] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1331 -g 10.240.45.178@tcp
[21354.813435] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1332 -g 10.240.45.178@tcp
[21354.819179] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1333 -g 10.240.45.178@tcp
[21354.824895] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1334 -g 10.240.45.178@tcp
[21354.830600] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1335 -g 10.240.45.178@tcp
[21354.836251] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1336 -g 10.240.45.178@tcp
[21354.841949] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1337 -g 10.240.45.178@tcp
[21354.847718] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1338 -g 10.240.45.178@tcp
[21354.853552] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1339 -g 10.240.45.178@tcp
[21354.859264] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1340 -g 10.240.45.178@tcp
[21354.864890] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1341 -g 10.240.45.178@tcp
[21354.870619] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1342 -g 10.240.45.178@tcp
[21354.876251] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1343 -g 10.240.45.178@tcp
[21354.881768] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1344 -g 10.240.45.178@tcp
[21354.887384] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1345 -g 10.240.45.178@tcp
[21354.893034] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1346 -g 10.240.45.178@tcp
[21354.898900] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1347 -g 10.240.45.178@tcp
[21354.904602] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1348 -g 10.240.45.178@tcp
[21354.910327] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1349 -g 10.240.45.178@tcp
[21354.916052] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1350 -g 10.240.45.178@tcp
[21354.921831] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1351 -g 10.240.45.178@tcp
[21354.927602] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1352 -g 10.240.45.178@tcp
[21354.933328] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1353 -g 10.240.45.178@tcp
[21354.939084] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1354 -g 10.240.45.178@tcp
[21354.944820] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1355 -g 10.240.45.178@tcp
[21354.950445] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1356 -g 10.240.45.178@tcp
[21354.956200] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1357 -g 10.240.45.178@tcp
[21354.962081] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1358 -g 10.240.45.178@tcp
[21354.967858] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1359 -g 10.240.45.178@tcp
[21354.973742] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1360 -g 10.240.45.178@tcp
[21354.979607] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1361 -g 10.240.45.178@tcp
[21354.985441] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1362 -g 10.240.45.178@tcp
[21354.991317] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1363 -g 10.240.45.178@tcp
[21354.997102] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1364 -g 10.240.45.178@tcp
[21355.003026] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1365 -g 10.240.45.178@tcp
[21355.008792] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1366 -g 10.240.45.178@tcp
[21355.014459] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1367 -g 10.240.45.178@tcp
[21355.020267] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1368 -g 10.240.45.178@tcp
[21355.025888] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1369 -g 10.240.45.178@tcp
[21355.031569] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1370 -g 10.240.45.178@tcp
[21355.037240] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1371 -g 10.240.45.178@tcp
[21355.042972] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1372 -g 10.240.45.178@tcp
[21355.048649] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1373 -g 10.240.45.178@tcp
[21355.054377] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1374 -g 10.240.45.178@tcp
[21355.060283] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1375 -g 10.240.45.178@tcp
[21355.065896] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1376 -g 10.240.45.178@tcp
[21355.071559] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1377 -g 10.240.45.178@tcp
[21355.077194] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1378 -g 10.240.45.178@tcp
[21355.083021] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1379 -g 10.240.45.178@tcp
[21355.088732] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1380 -g 10.240.45.178@tcp
[21355.094335] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1381 -g 10.240.45.178@tcp
[21355.100007] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1382 -g 10.240.45.178@tcp
[21355.105741] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1383 -g 10.240.45.178@tcp
[21355.111279] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1384 -g 10.240.45.178@tcp
[21355.116973] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1385 -g 10.240.45.178@tcp
[21355.122896] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1386 -g 10.240.45.178@tcp
[21355.128715] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1387 -g 10.240.45.178@tcp
[21355.134363] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1388 -g 10.240.45.178@tcp
[21355.140131] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1389 -g 10.240.45.178@tcp
[21355.145735] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1390 -g 10.240.45.178@tcp
[21355.151455] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1391 -g 10.240.45.178@tcp
[21355.157095] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1392 -g 10.240.45.178@tcp
[21355.162952] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1393 -g 10.240.45.178@tcp
[21355.168686] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1394 -g 10.240.45.178@tcp
[21355.174307] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1395 -g 10.240.45.178@tcp
[21355.179964] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1396 -g 10.240.45.178@tcp
[21355.185646] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1397 -g 10.240.45.178@tcp
[21355.191472] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1398 -g 10.240.45.178@tcp
[21355.197173] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1399 -g 10.240.45.178@tcp
[21355.202776] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1400 -g 10.240.45.178@tcp
[21355.208480] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1401 -g 10.240.45.178@tcp
[21355.213972] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1402 -g 10.240.45.178@tcp
[21355.219623] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1403 -g 10.240.45.178@tcp
[21355.225183] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1404 -g 10.240.45.178@tcp
[21355.230825] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1405 -g 10.240.45.178@tcp
[21355.236424] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1406 -g 10.240.45.178@tcp
[21355.242131] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1407 -g 10.240.45.178@tcp
[21355.247647] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1408 -g 10.240.45.178@tcp
[21355.253203] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1409 -g 10.240.45.178@tcp
[21355.258853] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1410 -g 10.240.45.178@tcp
[21355.264463] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1411 -g 10.240.45.178@tcp
[21355.269964] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1412 -g 10.240.45.178@tcp
[21355.275588] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1413 -g 10.240.45.178@tcp
[21355.281179] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1414 -g 10.240.45.178@tcp
[21355.286799] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1415 -g 10.240.45.178@tcp
[21355.292352] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1416 -g 10.240.45.178@tcp
[21355.298005] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1417 -g 10.240.45.178@tcp
[21355.303612] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1418 -g 10.240.45.178@tcp
[21355.309490] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1419 -g 10.240.45.178@tcp
[21355.315304] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1420 -g 10.240.45.178@tcp
[21355.321151] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1421 -g 10.240.45.178@tcp
[21355.326964] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1422 -g 10.240.45.178@tcp
[21355.332709] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1423 -g 10.240.45.178@tcp
[21355.338430] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1424 -g 10.240.45.178@tcp
[21355.344206] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1425 -g 10.240.45.178@tcp
[21355.350065] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1426 -g 10.240.45.178@tcp
[21355.356010] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1427 -g 10.240.45.178@tcp
[21355.361785] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1428 -g 10.240.45.178@tcp
[21355.367628] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1429 -g 10.240.45.178@tcp
[21355.373450] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1430 -g 10.240.45.178@tcp
[21355.379371] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1431 -g 10.240.45.178@tcp
[21355.385185] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1432 -g 10.240.45.178@tcp
[21355.391068] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1433 -g 10.240.45.178@tcp
[21355.396908] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1434 -g 10.240.45.178@tcp
[21355.402734] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1435 -g 10.240.45.178@tcp
[21355.408530] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1436 -g 10.240.45.178@tcp
[21355.414163] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1437 -g 10.240.45.178@tcp
[21355.419839] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1438 -g 10.240.45.178@tcp
[21355.425504] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1439 -g 10.240.45.178@tcp
[21355.431284] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1440 -g 10.240.45.178@tcp
[21355.437108] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1441 -g 10.240.45.178@tcp
[21355.442924] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1442 -g 10.240.45.178@tcp
[21355.448677] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1443 -g 10.240.45.178@tcp
[21355.454402] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1444 -g 10.240.45.178@tcp
[21355.460225] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1445 -g 10.240.45.178@tcp
[21355.466120] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1446 -g 10.240.45.178@tcp
[21355.471924] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1447 -g 10.240.45.178@tcp
[21355.477749] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1448 -g 10.240.45.178@tcp
[21355.483651] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1449 -g 10.240.45.178@tcp
[21355.489543] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1450 -g 10.240.45.178@tcp
[21355.495429] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1451 -g 10.240.45.178@tcp
[21355.501253] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1452 -g 10.240.45.178@tcp
[21355.507087] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1453 -g 10.240.45.178@tcp
[21355.512902] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1454 -g 10.240.45.178@tcp
[21355.518844] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1455 -g 10.240.45.178@tcp
[21355.529679] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1456 -g 10.240.45.178@tcp
[21355.535516] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1457 -g 10.240.45.178@tcp
[21355.541342] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1458 -g 10.240.45.178@tcp
[21355.547272] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1459 -g 10.240.45.178@tcp
[21355.553176] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1460 -g 10.240.45.178@tcp
[21355.558980] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1461 -g 10.240.45.178@tcp
[21355.564688] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1462 -g 10.240.45.178@tcp
[21355.570379] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1463 -g 10.240.45.178@tcp
[21355.576198] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1464 -g 10.240.45.178@tcp
[21355.582047] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1465 -g 10.240.45.178@tcp
[21355.588006] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1466 -g 10.240.45.178@tcp
[21355.593814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1467 -g 10.240.45.178@tcp
[21355.599618] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1468 -g 10.240.45.178@tcp
[21355.605402] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1469 -g 10.240.45.178@tcp
[21355.611358] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1470 -g 10.240.45.178@tcp
[21355.617206] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1471 -g 10.240.45.178@tcp
[21355.623117] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1472 -g 10.240.45.178@tcp
[21355.628975] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1473 -g 10.240.45.178@tcp
[21355.634882] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1474 -g 10.240.45.178@tcp
[21355.640813] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1475 -g 10.240.45.178@tcp
[21355.646549] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1476 -g 10.240.45.178@tcp
[21355.652333] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1477 -g 10.240.45.178@tcp
[21355.658268] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1478 -g 10.240.45.178@tcp
[21355.664187] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1479 -g 10.240.45.178@tcp
[21355.670010] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1480 -g 10.240.45.178@tcp
[21355.675869] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1481 -g 10.240.45.178@tcp
[21355.681806] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1482 -g 10.240.45.178@tcp
[21355.687623] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1483 -g 10.240.45.178@tcp
[21355.693456] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1484 -g 10.240.45.178@tcp
[21355.699357] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1485 -g 10.240.45.178@tcp
[21355.705162] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1486 -g 10.240.45.178@tcp
[21355.710976] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1487 -g 10.240.45.178@tcp
[21355.716779] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1488 -g 10.240.45.178@tcp
[21355.722439] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1489 -g 10.240.45.178@tcp
[21355.728185] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1490 -g 10.240.45.178@tcp
[21355.734016] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1491 -g 10.240.45.178@tcp
[21355.739863] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1492 -g 10.240.45.178@tcp
[21355.745702] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1493 -g 10.240.45.178@tcp
[21355.751444] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1494 -g 10.240.45.178@tcp
[21355.757370] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1495 -g 10.240.45.178@tcp
[21355.763118] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1496 -g 10.240.45.178@tcp
[21355.768915] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1497 -g 10.240.45.178@tcp
[21355.774773] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1498 -g 10.240.45.178@tcp
[21355.780493] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1499 -g 10.240.45.178@tcp
[21355.786423] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1500 -g 10.240.45.178@tcp
[21355.794639] LNetError: 828076:0:(router.c:333:lnet_is_route_alive()) peer 10.240.45.178@tcp is being used as a gateway but routing feature is not turned on
[21355.809783] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.45.178@tcp 2>/dev/null)"; if [[ -n "${output}" ]]; then echo "Delete route to tcp1 via 10.240.45.178@tcp"; /usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.45.178@tcp; e
[21356.392453] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21356.735081] LNet: 828182:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21357.774678] LNet: Removed LNI 10.240.44.153@tcp
[21357.871069] Key type .llcrypt unregistered
[21357.871904] Key type ._llcrypt unregistered
[21359.194443] Key type ._llcrypt registered
[21359.195316] Key type .llcrypt registered
[21359.433601] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 112: multiple net configurations ===== 19:38:33 \(1734377913\)
[21359.761472] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 19:38:33 (1734377913)
[21359.902329] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21359.902329] lctl dl | grep ' ST ' || true
[21359.958616] Key type .llcrypt unregistered
[21359.959440] Key type ._llcrypt unregistered
[21360.156581] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21360.157828] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21360.245556] Key type ._llcrypt registered
[21360.246375] Key type .llcrypt registered
[21360.329236] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21361.104298] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21361.113413] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21361.114522] LNet: Accept all, port 7988
[21361.205033] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180]
[21361.466013] LNet: 829282:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21361.467739] LNet: Removed LNI 10.240.44.153@tcp1
[21361.486641] LNet: Removed LNI 10.1.2.3@tcp2
[21361.493242] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21361.493242] lctl dl | grep ' ST ' || true
[21361.610673] Key type .llcrypt unregistered
[21361.611469] Key type ._llcrypt unregistered
[21363.222437] Key type ._llcrypt registered
[21363.223599] Key type .llcrypt registered
[21363.464485] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 199: load lnet w\/o module option, configure interface twice ========================================================== 19:38:37 \(1734377917\)
[21363.802393] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 19:38:37 (1734377917)
[21363.974712] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21364.741602] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21364.750528] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21364.754205] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21364.755589] LNet: Accept all, port 7988
[21364.853854] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --ip2net tcp 10.240.44.153
[21364.857285] LNet: Removed LNI 10.240.44.153@tcp
[21366.398085] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 200: load lnet w\/o module option, configure in a non-default namespace ========================================================== 19:38:40 \(1734377920\)
[21366.723346] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 19:38:40 (1734377920)
[21366.856462] LNet: 830505:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21366.870786] LNet: Removed LNI 10.240.44.153@tcp
[21366.877329] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21366.877329] lctl dl | grep ' ST ' || true
[21366.990626] Key type .llcrypt unregistered
[21366.991444] Key type ._llcrypt unregistered
[21367.191235] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21367.192504] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21367.279376] Key type ._llcrypt registered
[21367.280189] Key type .llcrypt registered
[21367.345378] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21368.127257] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21368.128375] LNet: Accept all, port 7988
[21369.734835] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 19:38:43 \(1734377923\)
[21370.048664] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 19:38:43 (1734377923)
[21370.186888] LNet: 831199:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21370.202787] LNet: Removed LNI 10.1.2.3@tcp
[21370.209365] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21370.209365] lctl dl | grep ' ST ' || true
[21370.334693] Key type .llcrypt unregistered
[21370.335533] Key type ._llcrypt unregistered
[21370.614078] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21370.615390] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21370.682608] Key type ._llcrypt registered
[21370.683740] Key type .llcrypt registered
[21370.718212] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21371.491464] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21371.492564] LNet: Accept all, port 7988
[21373.070838] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 19:38:47 \(1734377927\)
[21373.405657] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 19:38:47 (1734377927)
[21373.538517] LNet: 831860:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21373.546787] LNet: Removed LNI 10.1.2.3@tcp
[21373.553360] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21373.553360] lctl dl | grep ' ST ' || true
[21373.682646] Key type .llcrypt unregistered
[21373.683515] Key type ._llcrypt unregistered
[21373.856315] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21373.857561] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21373.950248] Key type ._llcrypt registered
[21373.955407] Key type .llcrypt registered
[21374.015237] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21374.792044] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21374.793157] LNet: Accept all, port 7988
[21376.366547] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 19:38:50 \(1734377930\)
[21376.702649] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 19:38:50 (1734377930)
[21376.835741] LNet: 832553:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21376.854851] LNet: Removed LNI 10.1.2.3@tcp
[21376.862178] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21376.862178] lctl dl | grep ' ST ' || true
[21376.986696] Key type .llcrypt unregistered
[21376.987521] Key type ._llcrypt unregistered
[21377.233990] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21377.235320] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21377.320178] Key type ._llcrypt registered
[21377.320999] Key type .llcrypt registered
[21377.383125] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21378.153016] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21378.161161] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21378.162198] LNet: Accept all, port 7988
[21379.981787] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 19:38:54 \(1734377934\)
[21380.310947] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 19:38:54 (1734377934)
[21380.500329] LNet: 833589:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21380.510756] LNet: Removed LNI 10.1.2.3@tcp
[21380.517314] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21380.517314] lctl dl | grep ' ST ' || true
[21380.630724] Key type .llcrypt unregistered
[21380.631532] Key type ._llcrypt unregistered
[21380.813115] Key type ._llcrypt registered
[21380.813934] Key type .llcrypt registered
[21380.869120] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21381.644938] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21381.649494] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21381.650677] LNet: Accept all, port 7988
[21386.181180] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21386.254058] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21386.257370] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21386.297381] LNet: 833891:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21386.308487] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21386.313993] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21386.370095] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21386.429930] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21386.435729] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21386.494277] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21386.545456] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21386.551046] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21386.608141] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21386.660117] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21386.665425] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21386.722145] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21386.773419] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21386.778993] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21386.835985] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21386.839953] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21386.841511] LNet: Skipped 4 previous similar messages
[21386.881523] LNet: 834231:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21386.883263] LNet: 834231:0:(api-ni.c:337:recovery_interval_set()) Skipped 4 previous similar messages
[21386.894302] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21386.899983] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21386.911450] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21386.911450] lctl dl | grep ' ST ' || true
[21386.938576] LNet: 834260:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21386.954740] LNet: Removed LNI 10.240.44.153@tcp
[21387.038764] Key type .llcrypt unregistered
[21387.039589] Key type ._llcrypt unregistered
[21391.878593] Key type ._llcrypt registered
[21391.883049] Key type .llcrypt registered
[21392.124497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 19:39:06 \(1734377946\)
[21392.452680] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 19:39:06 (1734377946)
[21392.650836] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21392.650836] lctl dl | grep ' ST ' || true
[21392.706709] Key type .llcrypt unregistered
[21392.707513] Key type ._llcrypt unregistered
[21392.903295] Key type ._llcrypt registered
[21392.907176] Key type .llcrypt registered
[21392.968150] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21393.736611] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21393.740833] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21393.741918] LNet: Accept all, port 7988
[21398.267949] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21402.426888] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21402.432369] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21402.435882] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21402.611960] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21402.615862] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21402.617497] LNet: 1 local NIs in recovery (showing 1): 10.240.44.153@tcp
[21402.660214] LNet: 835385:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21402.674204] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21402.681423] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21402.753518] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21402.811072] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21402.817075] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21402.894831] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21402.950852] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21402.956332] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21403.027916] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21403.081724] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21403.087329] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21403.158105] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21403.162002] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21403.163590] LNet: Skipped 11 previous similar messages
[21403.204818] LNet: 835681:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21403.206519] LNet: 835681:0:(api-ni.c:337:recovery_interval_set()) Skipped 3 previous similar messages
[21403.217701] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21403.223207] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21403.294296] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21403.347256] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21403.353011] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21406.278135] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21406.278135] lctl dl | grep ' ST ' || true
[21406.305675] LNet: 835822:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21406.307386] LNet: Removed LNI 10.240.44.153@tcp
[21406.322775] LNet: Removed LNI 10.240.44.153@tcp1
[21406.418729] Key type .llcrypt unregistered
[21406.419529] Key type ._llcrypt unregistered
[21410.044016] Key type ._llcrypt registered
[21410.044856] Key type .llcrypt registered
[21410.286576] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 19:39:24 \(1734377964\)
[21410.638108] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 19:39:24 (1734377964)
[21410.813866] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21410.813866] lctl dl | grep ' ST ' || true
[21410.870757] Key type .llcrypt unregistered
[21410.871579] Key type ._llcrypt unregistered
[21411.057035] Key type ._llcrypt registered
[21411.058876] Key type .llcrypt registered
[21411.107196] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21411.893092] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21411.897457] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21411.898549] LNet: Accept all, port 7988
[21416.382892] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21416.458461] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21416.461905] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21416.504682] LNet: 836734:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21416.516490] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21416.522236] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21416.581560] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21416.634864] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21416.640328] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21416.700208] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21416.755108] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21416.760848] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21416.772396] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21416.772396] lctl dl | grep ' ST ' || true
[21416.799726] LNet: 836899:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21416.822793] LNet: Removed LNI 10.240.44.153@tcp
[21416.906809] Key type .llcrypt unregistered
[21416.907631] Key type ._llcrypt unregistered
[21420.535293] Key type ._llcrypt registered
[21420.540998] Key type .llcrypt registered
[21420.785446] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 19:39:34 \(1734377974\)
[21421.111305] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 19:39:34 (1734377974)
[21421.305835] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21421.305835] lctl dl | grep ' ST ' || true
[21421.362797] Key type .llcrypt unregistered
[21421.363617] Key type ._llcrypt unregistered
[21421.562676] Key type ._llcrypt registered
[21421.563519] Key type .llcrypt registered
[21421.630466] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21422.404833] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21422.409429] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21422.410509] LNet: Accept all, port 7988
[21426.890251] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21431.192447] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21431.198220] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21431.201754] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21431.384141] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21431.387810] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21431.389432] LNet: 1 peer NIs in recovery (showing 1): 10.240.45.178@tcp
[21431.433504] LNet: 838023:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21431.445555] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21431.451379] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21431.459978] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21431.532238] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21431.588876] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21431.594858] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21431.603475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21431.673406] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.178@tcp
[21431.726054] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21431.731798] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21431.740273] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21433.642024] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21433.642024] lctl dl | grep ' ST ' || true
[21433.669510] LNet: 838243:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21434.686755] LNet: Removed LNI 10.240.44.153@tcp
[21434.702820] LNet: Removed LNI 10.240.44.153@tcp1
[21434.794877] Key type .llcrypt unregistered
[21434.795742] Key type ._llcrypt unregistered
[21438.477483] Key type ._llcrypt registered
[21438.478356] Key type .llcrypt registered
[21438.720534] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 19:39:52 \(1734377992\)
[21439.055019] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 19:39:52 (1734377992)
[21439.200115] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21439.200115] lctl dl | grep ' ST ' || true
[21439.258802] Key type .llcrypt unregistered
[21439.259589] Key type ._llcrypt unregistered
[21439.417537] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21439.418850] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21439.557809] Key type ._llcrypt registered
[21439.558674] Key type .llcrypt registered
[21439.616265] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21440.386352] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21440.387577] LNet: Accept all, port 7988
[21440.394437] LNet: 839035:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21440.410859] LNet: Removed LNI 10.240.44.153@tcp
[21440.424501] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21440.424501] lctl dl | grep ' ST ' || true
[21440.546893] Key type .llcrypt unregistered
[21440.547720] Key type ._llcrypt unregistered
[21440.772609] Key type ._llcrypt registered
[21440.773521] Key type .llcrypt registered
[21440.808297] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21441.589963] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21441.591135] LNet: Added LNI 10.1.2.3@tcp1 [8/256/0/180]
[21441.592149] LNet: Accept all, port 7988
[21441.599200] LNet: 839190:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21441.600827] LNet: Removed LNI 10.240.44.153@tcp
[21441.618833] LNet: Removed LNI 10.1.2.3@tcp1
[21441.632627] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21441.632627] lctl dl | grep ' ST ' || true
[21441.754821] Key type .llcrypt unregistered
[21441.755684] Key type ._llcrypt unregistered
[21441.930024] Key type ._llcrypt registered
[21441.932470] Key type .llcrypt registered
[21441.990885] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21442.763210] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21442.764336] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21442.765339] LNet: Accept all, port 7988
[21442.772478] LNet: 839345:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21442.774087] LNet: Removed LNI 10.240.44.153@tcp
[21442.786833] LNet: Removed LNI 10.1.2.3@tcp
[21442.800568] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21442.800568] lctl dl | grep ' ST ' || true
[21442.918884] Key type .llcrypt unregistered
[21442.919753] Key type ._llcrypt unregistered
[21443.149461] Key type ._llcrypt registered
[21443.150349] Key type .llcrypt registered
[21443.194490] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21443.975222] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21443.976371] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21443.977380] LNet: Accept all, port 7988
[21443.984547] LNet: 839501:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21443.986169] LNet: Removed LNI 10.240.44.153@tcp
[21443.998955] LNet: Removed LNI 10.1.2.3@tcp
[21444.012983] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21444.012983] lctl dl | grep ' ST ' || true
[21444.146898] Key type .llcrypt unregistered
[21444.147773] Key type ._llcrypt unregistered
[21444.412099] Key type ._llcrypt registered
[21444.419748] Key type .llcrypt registered
[21444.498542] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21445.274231] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21445.275369] LNetError: 839713:0:(config.c:1555:lnet_inet_select()) ksocklnd: failed to find interface test1pg
[21445.277081] LNetError: Error -100 starting up LNI tcp
[21445.286862] LNet: Removed LNI 10.240.44.153@tcp
[21445.300437] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21445.300437] lctl dl | grep ' ST ' || true
[21445.414822] Key type .llcrypt unregistered
[21445.415659] Key type ._llcrypt unregistered
[21446.954016] Key type ._llcrypt registered
[21446.954990] Key type .llcrypt registered
[21447.193889] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 19:40:01 \(1734378001\)
[21447.528452] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 19:40:01 (1734378001)
[21447.735814] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21447.735814] lctl dl | grep ' ST ' || true
[21447.794829] Key type .llcrypt unregistered
[21447.795653] Key type ._llcrypt unregistered
[21447.968893] Key type ._llcrypt registered
[21447.969722] Key type .llcrypt registered
[21448.040401] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21448.812402] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21448.816893] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21448.817984] LNet: Accept all, port 7988
[21453.346894] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21453.419748] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21453.423101] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21453.463685] LNet: 840616:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21453.474726] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21453.480170] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21453.489959] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21453.489959] lctl dl | grep ' ST ' || true
[21453.516836] LNet: 840644:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21453.534933] LNet: Removed LNI 10.240.44.153@tcp
[21453.626850] Key type .llcrypt unregistered
[21453.627668] Key type ._llcrypt unregistered
[21456.148732] Key type ._llcrypt registered
[21456.149603] Key type .llcrypt registered
[21456.209773] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21456.997361] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21457.001781] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21457.002923] LNet: Accept all, port 7988
[21461.498136] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21465.723006] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21465.728391] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21465.731872] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21465.903812] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.45.178@tcp
[21465.907622] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21465.949211] LNet: 841312:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[21465.960346] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[21465.965765] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[21468.895652] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21468.904242] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21468.904242] lctl dl | grep ' ST ' || true
[21468.931621] LNet: 841377:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21468.933294] LNet: Removed LNI 10.240.44.153@tcp
[21468.950906] LNet: Removed LNI 10.240.44.153@tcp1
[21469.042885] Key type .llcrypt unregistered
[21469.043688] Key type ._llcrypt unregistered
[21472.698501] Key type ._llcrypt registered
[21472.699480] Key type .llcrypt registered
[21472.942660] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 210: Local NI recovery checks ======== 19:40:27 \(1734378027\)
[21473.279294] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 19:40:27 (1734378027)
[21473.462374] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21474.229281] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21474.238056] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21474.241702] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21474.242785] LNet: Accept all, port 7988
[21474.335233] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21474.338622] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21474.443334] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.44.153@tcp
[21474.454000] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 10
[21474.470783] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 0 --nid 10.240.44.153@tcp
[21474.474362] LNetError: 842241:0:(api-ni.c:4061:lnet_ni_set_healthv()) manually adding local NI 10.240.44.153@tcp to recovery
[21474.480175] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21475.262913] LNet: 1 local NIs in recovery (showing 1): 10.240.44.153@tcp
[21475.262928] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21475.498310] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21476.516936] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21477.311117] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21477.535456] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21477.555903] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | grep -wc nid-0: 10.240.44.153@tcp
[21477.569020] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[21477.575679] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21478.593631] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21479.612649] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21480.631105] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21481.407077] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21481.650067] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21481.671012] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | grep -wc nid-0: 10.240.44.153@tcp
[21481.683177] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[21481.695646] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21481.698993] LNet: 842367:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21482.718897] LNet: Removed LNI 10.240.44.153@tcp
[21482.734933] LNet: Removed LNI 10.240.44.153@tcp1
[21482.739108] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21482.748415] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21482.752573] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21482.753638] LNet: Accept all, port 7988
[21482.851921] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21482.855691] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21482.966808] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.44.153@tcp
[21482.973490] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 0
[21482.984359] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 4
[21483.002320] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 0 --nid 10.240.44.153@tcp
[21483.006125] LNetError: 842628:0:(api-ni.c:4061:lnet_ni_set_healthv()) manually adding local NI 10.240.44.153@tcp to recovery
[21483.011762] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21483.774960] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21484.030554] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21485.049198] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21486.067651] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21486.087948] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | grep -wc nid-0: 10.240.44.153@tcp
[21486.099664] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[21486.106213] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21487.124008] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21488.142684] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21489.161406] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21489.918955] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21489.920579] LNet: Skipped 1 previous similar message
[21490.180544] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21491.199896] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21492.218991] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21493.238046] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21494.256901] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21494.277698] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | grep -wc nid-0: 10.240.44.153@tcp
[21494.289570] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[21494.297033] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 900
[21495.939583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 211: Remote NI recovery checks ======= 19:40:50 \(1734378050\)
[21496.277471] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 19:40:50 (1734378050)
[21496.417677] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21496.421061] LNet: 843240:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21496.422746] LNet: Removed LNI 10.240.44.153@tcp
[21496.434994] LNet: Removed LNI 10.240.44.153@tcp1
[21496.438338] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21496.446896] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21496.450428] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21496.451711] LNet: Accept all, port 7988
[21496.543967] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21496.547597] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21496.653392] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.44.153@tcp
[21496.664524] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 10
[21496.679375] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --nid 10.240.44.153@tcp --health 0
[21496.686480] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21496.698500] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[21496.705206] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21497.470928] LNet: 1 peer NIs in recovery (showing 1): 10.240.44.153@tcp
[21497.720360] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21498.494974] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21498.496667] LNet: Skipped 1 previous similar message
[21498.736123] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21499.751835] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21500.767595] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21501.783442] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21502.799071] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21503.814784] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21504.830502] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21505.846274] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21506.862173] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21507.878045] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21508.893891] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21509.909633] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21510.925251] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21511.940879] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21511.953409] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[21511.960046] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21511.984300] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 0
[21511.989576] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --nid 10.240.44.153@tcp --health 500
[21511.996462] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21512.008329] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[21512.014806] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21513.029831] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21514.045594] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21515.060811] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21515.516577] Autotest: Test running for 355 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[21516.081518] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21517.098704] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21518.114452] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21518.127127] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[21518.137282] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21518.140596] LNet: 843817:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21519.166959] LNet: Removed LNI 10.240.44.153@tcp
[21519.183055] LNet: Removed LNI 10.240.44.153@tcp1
[21519.187080] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21519.195914] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21519.199690] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21519.200767] LNet: Accept all, port 7988
[21519.293674] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21519.297247] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21519.403171] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.44.153@tcp
[21519.409487] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 0
[21519.419556] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 4
[21519.434452] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --nid 10.240.44.153@tcp --health 0
[21519.441533] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21520.223026] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21520.224629] LNet: Skipped 3 previous similar messages
[21520.461298] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21520.485902] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21520.498029] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[21520.504640] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21521.524248] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21522.544309] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21523.564308] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21524.584563] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21525.604995] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21526.625208] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21527.645834] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21528.665987] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21529.686279] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21530.706842] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.44.153@tcp | grep -v primary | grep -e 10.240.44.153@tcp -e ping_count | grep -wA1 10.240.44.153@tcp | awk '/ping_count/{print $NF}'
[21530.731471] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21530.743963] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[21530.751572] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 900
[21532.403918] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 212: Check discovery refcount loss bug \(LU-14627\) ========================================================== 19:41:26 \(1734378086\)
[21532.753166] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 19:41:26 (1734378086)
[21532.938447] LNet: 844757:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21532.940211] LNet: Removed LNI 10.240.44.153@tcp
[21532.963157] LNet: Removed LNI 10.240.44.153@tcp1
[21532.970492] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21532.970492] lctl dl | grep ' ST ' || true
[21533.083139] Key type .llcrypt unregistered
[21533.083950] Key type ._llcrypt unregistered
[21533.315488] Key type ._llcrypt registered
[21533.316346] Key type .llcrypt registered
[21533.377646] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21534.156799] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[21534.161236] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21534.162319] LNet: Accept all, port 7988
[21539.000738] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Initial discovery
[21539.334519] Lustre: DEBUG MARKER: Initial discovery
[21539.467783] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover --force 10.240.45.178@tcp
[21540.346559] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Fail local discover ping to set LNET_PEER_REDISCOVER flag
[21540.675331] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag
[21540.812383] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover --force 10.240.45.178@tcp
[21540.816195] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21541.082662] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Force trevis-106vm14.trevis.whamcloud.com to discover 10.240.45.178@tcp \(in background\)
[21541.421708] Lustre: DEBUG MARKER: Force trevis-106vm14.trevis.whamcloud.com to discover 10.240.45.178@tcp (in background)
[21541.559246] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover --force 10.240.45.178@tcp
[21541.810709] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Force trevis-127vm20 to discover 10.240.44.153@tcp
[21542.166778] Lustre: DEBUG MARKER: Force trevis-127vm20 to discover 10.240.44.153@tcp
[21543.147895] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for 845348
[21543.483037] Lustre: DEBUG MARKER: Wait for 845348
[21544.607047] LNet: There was an unexpected network error while writing to 10.240.45.178: rc = -22
[21544.862018] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Finished wait on 845348
[21545.198521] Lustre: DEBUG MARKER: Finished wait on 845348
[21545.340543] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21545.340543] lctl dl | grep ' ST ' || true
[21545.367863] LNet: 845717:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21546.415097] LNet: Removed LNI 10.240.44.153@tcp
[21546.511141] Key type .llcrypt unregistered
[21546.512027] Key type ._llcrypt unregistered
[21550.181582] Key type ._llcrypt registered
[21550.182400] Key type .llcrypt registered
[21550.426897] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 19:41:44 \(1734378104\)
[21550.752581] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 19:41:44 (1734378104)
[21550.887328] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21550.887328] lctl dl | grep ' ST ' || true
[21550.947059] Key type .llcrypt unregistered
[21550.947850] Key type ._llcrypt unregistered
[21551.197167] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[21551.198512] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[21551.301835] Key type ._llcrypt registered
[21551.302671] Key type .llcrypt registered
[21551.362806] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21552.148710] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21552.164828] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21552.168614] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21552.169736] LNet: Accept all, port 7988
[21552.260202] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[21552.263763] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21554.065670] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 19:41:48 \(1734378108\)
[21554.395134] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 19:41:48 (1734378108)
[21554.524449] LNet: 847345:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21554.526074] LNet: Removed LNI 10.240.44.153@tcp
[21554.539230] LNet: Removed LNI 10.1.2.3@tcp
[21554.546052] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21554.546052] lctl dl | grep ' ST ' || true
[21554.675093] Key type .llcrypt unregistered
[21554.675901] Key type ._llcrypt unregistered
[21554.864590] Key type ._llcrypt registered
[21554.865421] Key type .llcrypt registered
[21554.917202] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21555.693956] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21555.702815] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21555.706340] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21555.707537] LNet: Accept all, port 7988
[21555.795664] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[21555.799258] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[21555.927105] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.1.2.3@tcp 10.240.44.153@tcp
[21556.959053] LNetError: Unexpected error -22 connecting to 10.240.44.153@tcp at host 10.240.44.153:7988
[21557.526933] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 19:41:51 \(1734378111\)
[21557.876183] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 19:41:51 (1734378111)
[21557.994855] LNet: 848251:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21557.996662] LNet: Removed LNI 10.240.44.153@tcp
[21558.011237] LNet: Removed LNI 10.1.2.3@tcp
[21558.018133] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21558.018133] lctl dl | grep ' ST ' || true
[21558.147095] Key type .llcrypt unregistered
[21558.147961] Key type ._llcrypt unregistered
[21558.335260] Key type ._llcrypt registered
[21558.338643] Key type .llcrypt registered
[21558.395441] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21559.165088] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21559.174180] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21559.177811] LNet: 848402:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[21559.179743] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21559.180795] LNet: Accept all, port 7988
[21559.271522] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp2 --if eth0
[21559.274807] LNet: Added LNI 10.240.44.153@tcp2 [8/256/0/180]
[21559.383006] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim 10.240.44.153@tcp1 --nid 10.240.44.153@tcp2
[21560.010740] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.44.153@tcp1 10.240.44.153@tcp2
[21560.014543] LNetError: 848961:0:(lib-move.c:2224:lnet_handle_find_routed_path()) Unable to send message from 10.240.44.153@tcp1 to 10.240.44.153@tcp2 - Route table may be misconfigured
[21560.019510] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.44.153@tcp2 10.240.44.153@tcp1
[21561.554000] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 19:41:55 \(1734378115\)
[21561.892466] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 19:41:55 (1734378115)
[21562.029674] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21562.033183] LNet: 849407:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21562.034892] LNet: Removed LNI 10.240.44.153@tcp1
[21562.047166] LNet: Removed LNI 10.240.44.153@tcp2
[21562.050692] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21562.059558] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21562.063182] LNet: 849414:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[21562.064862] LNet: 849414:0:(lib-socket.c:567:lnet_inet_enumerate()) Skipped 1 previous similar message
[21562.066456] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21562.067777] LNet: Accept all, port 7988
[21562.158789] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21562.162148] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21562.269531] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.44.153@tcp
[21562.275803] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.44.153@tcp 10.240.44.153@tcp
[21562.281907] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.44.153@tcp1 10.240.44.153@tcp1
[21562.297136] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.44.153@tcp
[21562.300527] LNet: There was an unexpected network error while writing to 10.240.44.153: rc = -22
[21562.307967] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | grep -wc nid-0: 10.240.44.153@tcp
[21562.319952] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[21562.326476] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | grep -wc nid-0: 10.240.44.153@tcp
[21562.338548] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[21563.075094] LNet: 1 local NIs in recovery (showing 1): 10.240.44.153@tcp
[21563.880139] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 217: Don\'t leak memory when discovering peer with nnis \<= 1 ========================================================== 19:41:57 \(1734378117\)
[21564.214403] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 19:41:57 (1734378117)
[21564.344783] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21564.348179] LNet: 850130:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21564.349844] LNet: Removed LNI 10.240.44.153@tcp
[21564.359179] LNet: Removed LNI 10.240.44.153@tcp1
[21564.362647] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21564.375543] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 0@lo
[21564.384154] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21564.384154] lctl dl | grep ' ST ' || true
[21564.507249] Key type .llcrypt unregistered
[21564.508064] Key type ._llcrypt unregistered
[21566.002492] Key type ._llcrypt registered
[21566.003381] Key type .llcrypt registered
[21566.247176] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 19:42:00 \(1734378120\)
[21566.583387] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 19:42:00 (1734378120)
[21566.754245] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21567.522637] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21567.783628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-lnet test_218 Need two LNet interfaces
[21568.108256] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces
[21568.506538] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 219: Consolidate peer entries ======== 19:42:02 \(1734378122\)
[21568.843807] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 19:42:02 (1734378122)
[21568.979582] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[21568.982935] LNet: 850975:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21568.987113] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21568.995585] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21568.999314] LNet: 850982:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[21569.001073] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21569.002102] LNet: Accept all, port 7988
[21569.093755] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[21569.097122] LNet: Added LNI 10.240.44.153@tcp1 [8/256/0/180]
[21569.205119] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.44.153@tcp
[21569.211328] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.44.153@tcp1
[21569.217288] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.44.153@tcp1
[21570.742064] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 220: Add routes w\/default options - check aliveness ========================================================== 19:42:04 \(1734378124\)
[21571.068985] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 19:42:04 (1734378124)
[21580.907374] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[21581.257206] LNet: 852076:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21582.271099] LNet: Removed LNI 10.240.44.153@tcp
[21583.311225] LNet: Removed LNI 10.240.44.153@tcp1
[21583.395151] Key type .llcrypt unregistered
[21583.395985] Key type ._llcrypt unregistered
[21585.281767] Key type ._llcrypt registered
[21585.282623] Key type .llcrypt registered
[21585.513686] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21585.925869] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[21586.277616] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21587.883224] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21594.585196] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[21594.588892] LNet: 852650:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[21594.590696] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[21594.591945] LNet: Accept all, port 7988
[21596.265128] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.45.178@tcp
[21596.273520] LNet: 852809:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[21596.638111] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[21596.655039] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[21597.388486] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.45.50@tcp1
[21597.933054] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
[21598.915789] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
[21599.246333] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
[21600.690493] LNet: 853170:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21600.795470] LNet: 853170:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21600.899457] LNet: 853170:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21601.003483] LNet: 853170:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21601.107476] LNet: 853170:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21601.211495] LNet: 853170:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21601.315484] LNet: 853170:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21602.723499] Lustre: DEBUG MARKER: Start LST rw
[21602.869284] Lustre: DEBUG MARKER: Start LST rw
[21602.882562] LNet: 830667:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21602.990509] LNet: 830667:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[21628.321560] Lustre: DEBUG MARKER: lst stop brw_rw
[21682.335351] LNet: 853171:0:(rpc.c:1099:srpc_client_rpc_expired()) Client RPC expired: service 5, peer 12345-10.240.45.50@tcp1, timeout 64.
[21690.527393] LNet: 853171:0:(rpc.c:1099:srpc_client_rpc_expired()) Client RPC expired: service 5, peer 12345-10.240.44.153@tcp, timeout 64.
[21706.911438] LNet: 853171:0:(framework.c:246:sfw_session_expired()) Session expired! sid: 10.240.44.153@tcp-21602275, name: lnet_session
[21706.913649] LustreError: 797740:0:(brw_test.c:365:brw_server_rpc_done()) Bulk transfer from 12345-10.240.45.50@tcp1 has failed: -108
[21723.295472] LNet: 853171:0:(rpc.c:1099:srpc_client_rpc_expired()) Client RPC expired: service 0, peer 12345-10.240.45.50@tcp1, timeout 64.
[21751.455568] INFO: task tee:794047 blocked for more than 120 seconds.
[21751.456791] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[21751.457997] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21751.459325] task:tee state:D stack: 0 pid:794047 ppid: 3211 flags:0x00000002
[21751.460761] Call Trace:
[21751.461266] <TASK>
[21751.461741] __schedule+0x24e/0x590
[21751.462443] schedule+0x69/0x110
[21751.463058] io_schedule+0x46/0x80
[21751.463714] wait_on_page_bit_common+0x10c/0x3d0
[21751.464578] ? filemap_invalidate_unlock_two+0x50/0x50
[21751.465482] wait_on_page_bit+0x3f/0x50
[21751.466187] wait_on_page_writeback+0x26/0x80
[21751.466980] wait_for_stable_page+0x32/0x40
[21751.467748] grab_cache_page_write_begin+0x31/0x40
[21751.468626] nfs_write_begin+0x61/0x300 [nfs]
[21751.469551] generic_perform_write+0xc9/0x200
[21751.470346] ? __cond_resched+0x1a/0x50
[21751.471067] nfs_file_write+0x1a7/0x2c0 [nfs]
[21751.471891] new_sync_write+0x114/0x1a0
[21751.472623] vfs_write+0x1d5/0x270
[21751.473262] ksys_write+0x67/0xf0
[21751.473900] __x64_sys_write+0x19/0x20
[21751.474601] do_syscall_64+0x5c/0xc0
[21751.475272] ? vfs_write+0x1d5/0x270
[21751.475955] ? ksys_write+0xb5/0xf0
[21751.476619] ? exit_to_user_mode_prepare+0x37/0xb0
[21751.477517] ? syscall_exit_to_user_mode+0x35/0x50
[21751.478379] ? __x64_sys_write+0x19/0x20
[21751.479098] ? do_syscall_64+0x69/0xc0
[21751.479811] ? exit_to_user_mode_prepare+0x37/0xb0
[21751.480671] ? syscall_exit_to_user_mode+0x35/0x50
[21751.481530] ? __x64_sys_write+0x19/0x20
[21751.482246] ? do_syscall_64+0x69/0xc0
[21751.482940] ? do_syscall_64+0x69/0xc0
[21751.483649] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[21751.484555] RIP: 0033:0x7f07190b5887
[21751.485222] RSP: 002b:00007ffd458baa18 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[21751.486508] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007f07190b5887
[21751.487722] RDX: 000000000000001a RSI: 00007ffd458bab50 RDI: 0000000000000003
[21751.488932] RBP: 00007ffd458bab50 R08: 000000000000001a R09: 0000000000000001
[21751.490138] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[21751.491350] R13: 000055b4863b52a0 R14: 00007f07191b7a00 R15: 000000000000001a
[21751.492563] </TASK>
[21751.493026] INFO: task tee:851555 blocked for more than 120 seconds.
[21751.494135] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[21751.495304] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21751.496633] task:tee state:D stack: 0 pid:851555 ppid:794166 flags:0x00000002
[21751.498049] Call Trace:
[21751.498547] <TASK>
[21751.498984] __schedule+0x24e/0x590
[21751.499657] schedule+0x69/0x110
[21751.500276] io_schedule+0x46/0x80
[21751.500920] wait_on_page_bit_common+0x10c/0x3d0
[21751.501746] ? filemap_invalidate_unlock_two+0x50/0x50
[21751.502651] wait_on_page_bit+0x3f/0x50
[21751.503356] wait_on_page_writeback+0x26/0x80
[21751.504149] wait_for_stable_page+0x32/0x40
[21751.504917] grab_cache_page_write_begin+0x31/0x40
[21751.505788] nfs_write_begin+0x61/0x300 [nfs]
[21751.506601] generic_perform_write+0xc9/0x200
[21751.507392] ? __cond_resched+0x1a/0x50
[21751.508107] nfs_file_write+0x1a7/0x2c0 [nfs]
[21751.508922] new_sync_write+0x114/0x1a0
[21751.509640] vfs_write+0x1d5/0x270
[21751.510279] ksys_write+0x67/0xf0
[21751.510911] __x64_sys_write+0x19/0x20
[21751.511624] do_syscall_64+0x5c/0xc0
[21751.512294] ? do_syscall_64+0x69/0xc0
[21751.513006] ? exit_to_user_mode_prepare+0x96/0xb0
[21751.513861] ? syscall_exit_to_user_mode+0x35/0x50
[21751.514722] ? __x64_sys_read+0x19/0x20
[21751.515436] ? do_syscall_64+0x69/0xc0
[21751.516138] ? do_syscall_64+0x69/0xc0
[21751.516847] ? syscall_exit_to_user_mode+0x35/0x50
[21751.517712] ? __x64_sys_write+0x19/0x20
[21751.518441] ? do_syscall_64+0x69/0xc0
[21751.519139] ? do_syscall_64+0x69/0xc0
[21751.519856] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[21751.520766] RIP: 0033:0x7fcbf2bb4887
[21751.521441] RSP: 002b:00007ffd1b05dcc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[21751.522738] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007fcbf2bb4887
[21751.523957] RDX: 000000000000001a RSI: 00007ffd1b05de00 RDI: 0000000000000003
[21751.525170] RBP: 00007ffd1b05de00 R08: 000000000000001a R09: 0000000000000001
[21751.526379] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[21751.527591] R13: 000055c53b5ed2a0 R14: 00007fcbf2cb6a00 R15: 000000000000001a
[21751.528790] </TASK>
[21805.727678] nfs: server 10.240.32.204 not responding, still trying
[21805.727767] nfs: server 10.240.32.204 not responding, still trying
[21812.895739] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[21812.897285] LNetError: 852537:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.45.178@tcp has gone from up to down
[21813.407815] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21813.409839] LNetError: Unexpected error -2 connecting to 10.240.44.153@tcp at host 10.240.44.153:7988
[21813.919835] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[21813.921540] LNetError: Skipped 1 previous similar message
[21814.943732] nfs: server 10.240.32.204 not responding, timed out
[21815.967781] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[21815.969486] LNetError: Skipped 1 previous similar message
[21819.039800] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[21821.599867] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21824.159870] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[21824.161588] LNetError: Skipped 1 previous similar message
[21829.791838] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21834.399833] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[21834.401502] LNetError: Skipped 1 previous similar message
[21837.983852] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21846.175884] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21853.855923] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[21853.857591] LNetError: Skipped 1 previous similar message
[21854.367893] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21870.751941] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21870.753856] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) Skipped 1 previous similar message
[21872.287866] INFO: task tee:794047 blocked for more than 241 seconds.
[21872.289103] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[21872.290338] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21872.291675] task:tee state:D stack: 0 pid:794047 ppid: 3211 flags:0x00000002
[21872.293107] Call Trace:
[21872.293601] <TASK>
[21872.294046] __schedule+0x24e/0x590
[21872.294704] schedule+0x69/0x110
[21872.295309] io_schedule+0x46/0x80
[21872.295961] wait_on_page_bit_common+0x10c/0x3d0
[21872.296802] ? filemap_invalidate_unlock_two+0x50/0x50
[21872.297703] wait_on_page_bit+0x3f/0x50
[21872.298400] wait_on_page_writeback+0x26/0x80
[21872.299190] wait_for_stable_page+0x32/0x40
[21872.299958] grab_cache_page_write_begin+0x31/0x40
[21872.300808] nfs_write_begin+0x61/0x300 [nfs]
[21872.301612] generic_perform_write+0xc9/0x200
[21872.302395] ? __cond_resched+0x1a/0x50
[21872.303100] nfs_file_write+0x1a7/0x2c0 [nfs]
[21872.303901] new_sync_write+0x114/0x1a0
[21872.304605] vfs_write+0x1d5/0x270
[21872.305243] ksys_write+0x67/0xf0
[21872.305865] __x64_sys_write+0x19/0x20
[21872.306544] do_syscall_64+0x5c/0xc0
[21872.307206] ? vfs_write+0x1d5/0x270
[21872.307876] ? ksys_write+0xb5/0xf0
[21872.308523] ? exit_to_user_mode_prepare+0x37/0xb0
[21872.309369] ? syscall_exit_to_user_mode+0x35/0x50
[21872.310215] ? __x64_sys_write+0x19/0x20
[21872.310929] ? do_syscall_64+0x69/0xc0
[21872.311609] ? exit_to_user_mode_prepare+0x37/0xb0
[21872.312462] ? syscall_exit_to_user_mode+0x35/0x50
[21872.313317] ? __x64_sys_write+0x19/0x20
[21872.314030] ? do_syscall_64+0x69/0xc0
[21872.314712] ? do_syscall_64+0x69/0xc0
[21872.315387] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[21872.316272] RIP: 0033:0x7f07190b5887
[21872.316933] RSP: 002b:00007ffd458baa18 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[21872.318199] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007f07190b5887
[21872.319388] RDX: 000000000000001a RSI: 00007ffd458bab50 RDI: 0000000000000003
[21872.320586] RBP: 00007ffd458bab50 R08: 000000000000001a R09: 0000000000000001
[21872.321776] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[21872.322965] R13: 000055b4863b52a0 R14: 00007f07191b7a00 R15: 000000000000001a
[21872.324162] </TASK>
[21872.324614] INFO: task tee:851555 blocked for more than 241 seconds.
[21872.325687] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[21872.326862] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21872.328183] task:tee state:D stack: 0 pid:851555 ppid:794166 flags:0x00000002
[21872.329581] Call Trace:
[21872.330080] <TASK>
[21872.330518] __schedule+0x24e/0x590
[21872.331170] schedule+0x69/0x110
[21872.331771] io_schedule+0x46/0x80
[21872.332420] wait_on_page_bit_common+0x10c/0x3d0
[21872.333248] ? filemap_invalidate_unlock_two+0x50/0x50
[21872.334154] wait_on_page_bit+0x3f/0x50
[21872.334858] wait_on_page_writeback+0x26/0x80
[21872.335649] wait_for_stable_page+0x32/0x40
[21872.336443] grab_cache_page_write_begin+0x31/0x40
[21872.337298] nfs_write_begin+0x61/0x300 [nfs]
[21872.338096] generic_perform_write+0xc9/0x200
[21872.338876] ? __cond_resched+0x1a/0x50
[21872.339570] nfs_file_write+0x1a7/0x2c0 [nfs]
[21872.340373] new_sync_write+0x114/0x1a0
[21872.341083] vfs_write+0x1d5/0x270
[21872.341720] ksys_write+0x67/0xf0
[21872.342337] __x64_sys_write+0x19/0x20
[21872.343039] do_syscall_64+0x5c/0xc0
[21872.343706] ? do_syscall_64+0x69/0xc0
[21872.344398] ? exit_to_user_mode_prepare+0x96/0xb0
[21872.345249] ? syscall_exit_to_user_mode+0x35/0x50
[21872.346115] ? __x64_sys_read+0x19/0x20
[21872.346818] ? do_syscall_64+0x69/0xc0
[21872.347499] ? do_syscall_64+0x69/0xc0
[21872.348205] ? syscall_exit_to_user_mode+0x35/0x50
[21872.349054] ? __x64_sys_write+0x19/0x20
[21872.349769] ? do_syscall_64+0x69/0xc0
[21872.350452] ? do_syscall_64+0x69/0xc0
[21872.351154] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[21872.352052] RIP: 0033:0x7fcbf2bb4887
[21872.352716] RSP: 002b:00007ffd1b05dcc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[21872.353997] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007fcbf2bb4887
[21872.355195] RDX: 000000000000001a RSI: 00007ffd1b05de00 RDI: 0000000000000003
[21872.356394] RBP: 00007ffd1b05de00 R08: 000000000000001a R09: 0000000000000001
[21872.357593] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[21872.358783] R13: 000055c53b5ed2a0 R14: 00007fcbf2cb6a00 R15: 000000000000001a
[21872.359989] </TASK>
[21873.311851] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[21888.671999] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[21888.673712] LNetError: Skipped 1 previous similar message
[21903.520022] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[21903.521981] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) Skipped 3 previous similar messages
[21993.120157] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[21993.120157] INFO: task tee:794047 blocked for more than 362 seconds.
[21993.120162] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[21993.123993] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21993.125324] task:tee state:D stack: 0 pid:794047 ppid: 3211 flags:0x00000002
[21993.126731] Call Trace:
[21993.127233] <TASK>
[21993.127681] __schedule+0x24e/0x590
[21993.128349] schedule+0x69/0x110
[21993.128954] io_schedule+0x46/0x80
[21993.129586] wait_on_page_bit_common+0x10c/0x3d0
[21993.130403] ? filemap_invalidate_unlock_two+0x50/0x50
[21993.131295] wait_on_page_bit+0x3f/0x50
[21993.132001] wait_on_page_writeback+0x26/0x80
[21993.132784] wait_for_stable_page+0x32/0x40
[21993.133538] grab_cache_page_write_begin+0x31/0x40
[21993.134387] nfs_write_begin+0x61/0x300 [nfs]
[21993.135180] generic_perform_write+0xc9/0x200
[21993.135955] ? __cond_resched+0x1a/0x50
[21993.136662] nfs_file_write+0x1a7/0x2c0 [nfs]
[21993.137471] new_sync_write+0x114/0x1a0
[21993.138173] vfs_write+0x1d5/0x270
[21993.138799] ksys_write+0x67/0xf0
[21993.139423] __x64_sys_write+0x19/0x20
[21993.140122] do_syscall_64+0x5c/0xc0
[21993.140782] ? vfs_write+0x1d5/0x270
[21993.141453] ? ksys_write+0xb5/0xf0
[21993.142106] ? exit_to_user_mode_prepare+0x37/0xb0
[21993.142951] ? syscall_exit_to_user_mode+0x35/0x50
[21993.143803] ? __x64_sys_write+0x19/0x20
[21993.144528] ? do_syscall_64+0x69/0xc0
[21993.145220] ? exit_to_user_mode_prepare+0x37/0xb0
[21993.146071] ? syscall_exit_to_user_mode+0x35/0x50
[21993.146907] ? __x64_sys_write+0x19/0x20
[21993.147623] ? do_syscall_64+0x69/0xc0
[21993.148320] ? do_syscall_64+0x69/0xc0
[21993.149011] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[21993.149883] RIP: 0033:0x7f07190b5887
[21993.150543] RSP: 002b:00007ffd458baa18 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[21993.151818] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007f07190b5887
[21993.153020] RDX: 000000000000001a RSI: 00007ffd458bab50 RDI: 0000000000000003
[21993.154217] RBP: 00007ffd458bab50 R08: 000000000000001a R09: 0000000000000001
[21993.155405] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[21993.156614] R13: 000055b4863b52a0 R14: 00007f07191b7a00 R15: 000000000000001a
[21993.157821] </TASK>
[21993.158275] INFO: task tee:851555 blocked for more than 362 seconds.
[21993.159352] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[21993.160526] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21993.161829] task:tee state:D stack: 0 pid:851555 ppid:794166 flags:0x00000002
[21993.163234] Call Trace:
[21993.163720] <TASK>
[21993.164170] __schedule+0x24e/0x590
[21993.164816] schedule+0x69/0x110
[21993.165428] io_schedule+0x46/0x80
[21993.166063] wait_on_page_bit_common+0x10c/0x3d0
[21993.166877] ? filemap_invalidate_unlock_two+0x50/0x50
[21993.167769] wait_on_page_bit+0x3f/0x50
[21993.168473] wait_on_page_writeback+0x26/0x80
[21993.169253] wait_for_stable_page+0x32/0x40
[21993.170007] grab_cache_page_write_begin+0x31/0x40
[21993.170850] nfs_write_begin+0x61/0x300 [nfs]
[21993.171646] generic_perform_write+0xc9/0x200
[21993.172437] ? __cond_resched+0x1a/0x50
[21993.173146] nfs_file_write+0x1a7/0x2c0 [nfs]
[21993.173934] new_sync_write+0x114/0x1a0
[21993.174637] vfs_write+0x1d5/0x270
[21993.175280] ksys_write+0x67/0xf0
[21993.175896] __x64_sys_write+0x19/0x20
[21993.176594] do_syscall_64+0x5c/0xc0
[21993.177258] ? do_syscall_64+0x69/0xc0
[21993.177941] ? exit_to_user_mode_prepare+0x96/0xb0
[21993.178780] ? syscall_exit_to_user_mode+0x35/0x50
[21993.179628] ? __x64_sys_read+0x19/0x20
[21993.180343] ? do_syscall_64+0x69/0xc0
[21993.181034] ? do_syscall_64+0x69/0xc0
[21993.181711] ? syscall_exit_to_user_mode+0x35/0x50
[21993.182550] ? __x64_sys_write+0x19/0x20
[21993.183263] ? do_syscall_64+0x69/0xc0
[21993.183945] ? do_syscall_64+0x69/0xc0
[21993.184642] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[21993.185534] RIP: 0033:0x7fcbf2bb4887
[21993.186205] RSP: 002b:00007ffd1b05dcc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[21993.187483] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007fcbf2bb4887
[21993.188682] RDX: 000000000000001a RSI: 00007ffd1b05de00 RDI: 0000000000000003
[21993.189869] RBP: 00007ffd1b05de00 R08: 000000000000001a R09: 0000000000000001
[21993.191053] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[21993.192255] R13: 000055c53b5ed2a0 R14: 00007fcbf2cb6a00 R15: 000000000000001a
[21993.193449] </TASK>
[21995.552198] nfs: server 10.240.32.204 not responding, timed out
[22000.544171] nfs: server 10.240.32.204 not responding, timed out
[22010.528295] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[22010.529985] LNetError: Skipped 2 previous similar messages
[22113.952505] INFO: task tee:794047 blocked for more than 483 seconds.
[22113.953749] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[22113.954969] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[22113.956336] task:tee state:D stack: 0 pid:794047 ppid: 3211 flags:0x00000002
[22113.957792] Call Trace:
[22113.958322] <TASK>
[22113.958780] __schedule+0x24e/0x590
[22113.959450] schedule+0x69/0x110
[22113.960070] io_schedule+0x46/0x80
[22113.960735] wait_on_page_bit_common+0x10c/0x3d0
[22113.961578] ? filemap_invalidate_unlock_two+0x50/0x50
[22113.962501] wait_on_page_bit+0x3f/0x50
[22113.963218] wait_on_page_writeback+0x26/0x80
[22113.964019] wait_for_stable_page+0x32/0x40
[22113.964799] grab_cache_page_write_begin+0x31/0x40
[22113.965670] nfs_write_begin+0x61/0x300 [nfs]
[22113.966498] generic_perform_write+0xc9/0x200
[22113.967294] ? __cond_resched+0x1a/0x50
[22113.968012] nfs_file_write+0x1a7/0x2c0 [nfs]
[22113.968833] new_sync_write+0x114/0x1a0
[22113.969553] vfs_write+0x1d5/0x270
[22113.970195] ksys_write+0x67/0xf0
[22113.970839] __x64_sys_write+0x19/0x20
[22113.971541] do_syscall_64+0x5c/0xc0
[22113.972216] ? vfs_write+0x1d5/0x270
[22113.972903] ? ksys_write+0xb5/0xf0
[22113.973570] ? exit_to_user_mode_prepare+0x37/0xb0
[22113.974431] ? syscall_exit_to_user_mode+0x35/0x50
[22113.975292] ? __x64_sys_write+0x19/0x20
[22113.976013] ? do_syscall_64+0x69/0xc0
[22113.976723] ? exit_to_user_mode_prepare+0x37/0xb0
[22113.977587] ? syscall_exit_to_user_mode+0x35/0x50
[22113.978452] ? __x64_sys_write+0x19/0x20
[22113.979167] ? do_syscall_64+0x69/0xc0
[22113.979866] ? do_syscall_64+0x69/0xc0
[22113.980570] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[22113.981476] RIP: 0033:0x7f07190b5887
[22113.982153] RSP: 002b:00007ffd458baa18 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[22113.983462] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007f07190b5887
[22113.984690] RDX: 000000000000001a RSI: 00007ffd458bab50 RDI: 0000000000000003
[22113.985916] RBP: 00007ffd458bab50 R08: 000000000000001a R09: 0000000000000001
[22113.987125] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[22113.988348] R13: 000055b4863b52a0 R14: 00007f07191b7a00 R15: 000000000000001a
[22113.989573] </TASK>
[22113.990034] INFO: task tee:851555 blocked for more than 483 seconds.
[22113.991119] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[22113.992305] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[22113.993628] task:tee state:D stack: 0 pid:851555 ppid:794166 flags:0x00000002
[22113.995060] Call Trace:
[22113.995554] <TASK>
[22113.995996] __schedule+0x24e/0x590
[22113.996668] schedule+0x69/0x110
[22113.997295] io_schedule+0x46/0x80
[22113.997937] wait_on_page_bit_common+0x10c/0x3d0
[22113.998771] ? filemap_invalidate_unlock_two+0x50/0x50
[22113.999685] wait_on_page_bit+0x3f/0x50
[22114.000398] wait_on_page_writeback+0x26/0x80
[22114.001191] wait_for_stable_page+0x32/0x40
[22114.001954] grab_cache_page_write_begin+0x31/0x40
[22114.002805] nfs_write_begin+0x61/0x300 [nfs]
[22114.003612] generic_perform_write+0xc9/0x200
[22114.004398] ? __cond_resched+0x1a/0x50
[22114.005106] nfs_file_write+0x1a7/0x2c0 [nfs]
[22114.005911] new_sync_write+0x114/0x1a0
[22114.006617] vfs_write+0x1d5/0x270
[22114.007251] ksys_write+0x67/0xf0
[22114.007888] __x64_sys_write+0x19/0x20
[22114.008597] do_syscall_64+0x5c/0xc0
[22114.009276] ? do_syscall_64+0x69/0xc0
[22114.009972] ? exit_to_user_mode_prepare+0x96/0xb0
[22114.010821] ? syscall_exit_to_user_mode+0x35/0x50
[22114.011676] ? __x64_sys_read+0x19/0x20
[22114.012381] ? do_syscall_64+0x69/0xc0
[22114.013080] ? do_syscall_64+0x69/0xc0
[22114.013784] ? syscall_exit_to_user_mode+0x35/0x50
[22114.014647] ? __x64_sys_write+0x19/0x20
[22114.015365] ? do_syscall_64+0x69/0xc0
[22114.016056] ? do_syscall_64+0x69/0xc0
[22114.016761] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[22114.017663] RIP: 0033:0x7fcbf2bb4887
[22114.018326] RSP: 002b:00007ffd1b05dcc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[22114.019612] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007fcbf2bb4887
[22114.020829] RDX: 000000000000001a RSI: 00007ffd1b05de00 RDI: 0000000000000003
[22114.022038] RBP: 00007ffd1b05de00 R08: 000000000000001a R09: 0000000000000001
[22114.023248] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[22114.024463] R13: 000055c53b5ed2a0 R14: 00007fcbf2cb6a00 R15: 000000000000001a
[22114.025683] </TASK>
[22173.344601] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[22176.672616] nfs: server 10.240.32.204 not responding, timed out
[22185.824637] nfs: server 10.240.32.204 not responding, timed out
[22189.728828] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[22189.730639] LNetError: Skipped 3 previous similar messages
[22234.784775] INFO: task tee:794047 blocked for more than 604 seconds.
[22234.785995] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[22234.787196] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[22234.788528] task:tee state:D stack: 0 pid:794047 ppid: 3211 flags:0x00000002
[22234.789979] Call Trace:
[22234.790711] <TASK>
[22234.791417] __schedule+0x24e/0x590
[22234.792479] schedule+0x69/0x110
[22234.793413] io_schedule+0x46/0x80
[22234.794358] wait_on_page_bit_common+0x10c/0x3d0
[22234.795606] ? filemap_invalidate_unlock_two+0x50/0x50
[22234.796963] wait_on_page_bit+0x3f/0x50
[22234.798024] wait_on_page_writeback+0x26/0x80
[22234.799202] wait_for_stable_page+0x32/0x40
[22234.800134] grab_cache_page_write_begin+0x31/0x40
[22234.801027] nfs_write_begin+0x61/0x300 [nfs]
[22234.801960] generic_perform_write+0xc9/0x200
[22234.802752] ? __cond_resched+0x1a/0x50
[22234.803458] nfs_file_write+0x1a7/0x2c0 [nfs]
[22234.804263] new_sync_write+0x114/0x1a0
[22234.805004] vfs_write+0x1d5/0x270
[22234.805793] ksys_write+0x67/0xf0
[22234.806420] __x64_sys_write+0x19/0x20
[22234.807117] do_syscall_64+0x5c/0xc0
[22234.807790] ? vfs_write+0x1d5/0x270
[22234.808454] ? ksys_write+0xb5/0xf0
[22234.809152] ? exit_to_user_mode_prepare+0x37/0xb0
[22234.810181] ? syscall_exit_to_user_mode+0x35/0x50
[22234.811048] ? __x64_sys_write+0x19/0x20
[22234.811775] ? do_syscall_64+0x69/0xc0
[22234.812467] ? exit_to_user_mode_prepare+0x37/0xb0
[22234.813351] ? syscall_exit_to_user_mode+0x35/0x50
[22234.814333] ? __x64_sys_write+0x19/0x20
[22234.815064] ? do_syscall_64+0x69/0xc0
[22234.815766] ? do_syscall_64+0x69/0xc0
[22234.816467] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[22234.817386] RIP: 0033:0x7f07190b5887
[22234.818197] RSP: 002b:00007ffd458baa18 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[22234.819476] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007f07190b5887
[22234.820680] RDX: 000000000000001a RSI: 00007ffd458bab50 RDI: 0000000000000003
[22234.821899] RBP: 00007ffd458bab50 R08: 000000000000001a R09: 0000000000000001
[22234.823243] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[22234.824459] R13: 000055b4863b52a0 R14: 00007f07191b7a00 R15: 000000000000001a
[22234.825693] </TASK>
[22234.826252] INFO: task tee:851555 blocked for more than 604 seconds.
[22234.827344] Tainted: G W OE 5.15.0-94-generic #104-Ubuntu
[22234.828519] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[22234.829866] task:tee state:D stack: 0 pid:851555 ppid:794166 flags:0x00000002
[22234.831415] Call Trace:
[22234.831920] <TASK>
[22234.832360] __schedule+0x24e/0x590
[22234.833048] schedule+0x69/0x110
[22234.833775] io_schedule+0x46/0x80
[22234.834423] wait_on_page_bit_common+0x10c/0x3d0
[22234.835252] ? filemap_invalidate_unlock_two+0x50/0x50
[22234.836160] wait_on_page_bit+0x3f/0x50
[22234.836891] wait_on_page_writeback+0x26/0x80
[22234.837816] wait_for_stable_page+0x32/0x40
[22234.838581] grab_cache_page_write_begin+0x31/0x40
[22234.839431] nfs_write_begin+0x61/0x300 [nfs]
[22234.840230] generic_perform_write+0xc9/0x200
[22234.841071] ? __cond_resched+0x1a/0x50
[22234.841904] nfs_file_write+0x1a7/0x2c0 [nfs]
[22234.842701] new_sync_write+0x114/0x1a0
[22234.843399] vfs_write+0x1d5/0x270
[22234.844037] ksys_write+0x67/0xf0
[22234.844659] __x64_sys_write+0x19/0x20
[22234.845377] do_syscall_64+0x5c/0xc0
[22234.846178] ? do_syscall_64+0x69/0xc0
[22234.846878] ? exit_to_user_mode_prepare+0x96/0xb0
[22234.847729] ? syscall_exit_to_user_mode+0x35/0x50
[22234.848574] ? __x64_sys_read+0x19/0x20
[22234.849306] ? do_syscall_64+0x69/0xc0
[22234.850127] ? do_syscall_64+0x69/0xc0
[22234.850827] ? syscall_exit_to_user_mode+0x35/0x50
[22234.851688] ? __x64_sys_write+0x19/0x20
[22234.852401] ? do_syscall_64+0x69/0xc0
[22234.853129] ? do_syscall_64+0x69/0xc0
[22234.853978] entry_SYSCALL_64_after_hwframe+0x62/0xcc
[22234.854871] RIP: 0033:0x7fcbf2bb4887
[22234.855523] RSP: 002b:00007ffd1b05dcc8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[22234.856819] RAX: ffffffffffffffda RBX: 000000000000001a RCX: 00007fcbf2bb4887
[22234.858150] RDX: 000000000000001a RSI: 00007ffd1b05de00 RDI: 0000000000000003
[22234.859353] RBP: 00007ffd1b05de00 R08: 000000000000001a R09: 0000000000000001
[22234.860548] R10: 00000000000001b6 R11: 0000000000000246 R12: 000000000000001a
[22234.861862] R13: 000055c53b5ed2a0 R14: 00007fcbf2cb6a00 R15: 000000000000001a
[22234.863175] </TASK>
[22357.281088] nfs: server 10.240.32.204 not responding, timed out
[22362.273119] nfs: server 10.240.32.204 not responding, timed out
[22371.297107] nfs: server 10.240.32.204 not responding, timed out
[22412.961212] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[22491.809508] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[22491.811203] LNetError: Skipped 5 previous similar messages
[22537.761528] nfs: server 10.240.32.204 not responding, timed out
[22547.809578] nfs: server 10.240.32.204 not responding, timed out
[22556.833578] nfs: server 10.240.32.204 not responding, timed out
[22580.902454] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.45.50@tcp1 from <?>
[22580.904424] LNetError: 852537:0:(lib-move.c:2324:lnet_handle_find_routed_path()) Skipped 3 previous similar messages
[22580.909888] Lustre: DEBUG MARKER: Stop LST rw
[22712.993993] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[22718.049976] nfs: server 10.240.32.204 not responding, timed out
[22724.065990] nfs: server 10.240.32.204 not responding, timed out
[22733.234013] nfs: server 10.240.32.204 not responding, timed out
[22742.194046] nfs: server 10.240.32.204 not responding, timed out
[22761.538114] nfs: server 10.240.32.204 not responding, still trying
[22898.530425] nfs: server 10.240.32.204 not responding, timed out
[22903.586438] nfs: server 10.240.32.204 not responding, timed out
[22909.602458] nfs: server 10.240.32.204 not responding, timed out
[22918.626488] nfs: server 10.240.32.204 not responding, timed out
[22927.650495] nfs: server 10.240.32.204 not responding, timed out
[23013.026730] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[23039.650938] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[23039.652650] LNetError: Skipped 10 previous similar messages
[23079.010908] nfs: server 10.240.32.204 not responding, timed out
[23089.122930] nfs: server 10.240.32.204 not responding, timed out
[23095.138937] nfs: server 10.240.32.204 not responding, timed out
[23104.162979] nfs: server 10.240.32.204 not responding, timed out
[23113.186972] nfs: server 10.240.32.204 not responding, timed out
[23259.491365] nfs: server 10.240.32.204 not responding, timed out
[23264.675424] nfs: server 10.240.32.204 not responding, timed out
[23274.531386] nfs: server 10.240.32.204 not responding, timed out
[23280.567381] nfs: server 10.240.32.204 not responding, timed out
[23289.571433] nfs: server 10.240.32.204 not responding, timed out
[23298.595439] nfs: server 10.240.32.204 not responding, timed out
[23313.059497] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[23439.971813] nfs: server 10.240.32.204 not responding, timed out
[23445.027852] nfs: server 10.240.32.204 not responding, timed out
[23450.115836] nfs: server 10.240.32.204 not responding, timed out
[23460.067854] nfs: server 10.240.32.204 not responding, timed out
[23466.083863] nfs: server 10.240.32.204 not responding, timed out
[23475.107908] nfs: server 10.240.32.204 not responding, timed out
[23484.131915] nfs: server 10.240.32.204 not responding, timed out
[23613.092273] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[23620.452270] nfs: server 10.240.32.204 not responding, timed out
[23635.492311] nfs: server 10.240.32.204 not responding, timed out
[23645.604342] nfs: server 10.240.32.204 not responding, timed out
[23645.860444] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[23645.862099] LNetError: Skipped 9 previous similar messages
[23651.620345] nfs: server 10.240.32.204 not responding, timed out
[23660.644366] nfs: server 10.240.32.204 not responding, timed out
[23669.668393] nfs: server 10.240.32.204 not responding, timed out
[23800.932716] nfs: server 10.240.32.204 not responding, timed out
[23806.116749] nfs: server 10.240.32.204 not responding, timed out
[23821.156773] nfs: server 10.240.32.204 not responding, timed out
[23831.012813] nfs: server 10.240.32.204 not responding, timed out
[23837.108796] nfs: server 10.240.32.204 not responding, timed out
[23846.068826] nfs: server 10.240.32.204 not responding, timed out
[23855.076878] nfs: server 10.240.32.204 not responding, timed out
[23913.125006] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[23981.413190] nfs: server 10.240.32.204 not responding, timed out
[23986.597200] nfs: server 10.240.32.204 not responding, timed out
[23992.613196] nfs: server 10.240.32.204 not responding, timed out
[24008.485245] nfs: server 10.240.32.204 not responding, timed out
[24017.509281] nfs: server 10.240.32.204 not responding, timed out
[24023.525323] nfs: server 10.240.32.204 not responding, timed out
[24032.549343] nfs: server 10.240.32.204 not responding, timed out
[24041.573328] nfs: server 10.240.32.204 not responding, timed out
[24164.069645] nfs: server 10.240.32.204 not responding, timed out
[24173.093668] nfs: server 10.240.32.204 not responding, timed out
[24179.941702] nfs: server 10.240.32.204 not responding, timed out
[24194.981751] nfs: server 10.240.32.204 not responding, timed out
[24204.005731] nfs: server 10.240.32.204 not responding, timed out
[24210.021774] nfs: server 10.240.32.204 not responding, timed out
[24213.157795] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[24219.045780] nfs: server 10.240.32.204 not responding, timed out
[24227.237809] nfs: server 10.240.32.204 not responding, timed out
[24247.973967] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[24247.975643] LNetError: Skipped 10 previous similar messages
[24344.550138] nfs: server 10.240.32.204 not responding, timed out
[24351.398174] nfs: server 10.240.32.204 not responding, timed out
[24360.422139] nfs: server 10.240.32.204 not responding, timed out
[24366.438194] nfs: server 10.240.32.204 not responding, timed out
[24381.478199] nfs: server 10.240.32.204 not responding, timed out
[24390.502223] nfs: server 10.240.32.204 not responding, timed out
[24395.686247] nfs: server 10.240.32.204 not responding, timed out
[24404.710247] nfs: server 10.240.32.204 not responding, timed out
[24414.566276] nfs: server 10.240.32.204 not responding, timed out
[24513.190525] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[24525.862569] nfs: server 10.240.32.204 not responding, timed out
[24531.878580] nfs: server 10.240.32.204 not responding, timed out
[24537.894653] nfs: server 10.240.32.204 not responding, timed out
[24546.918630] nfs: server 10.240.32.204 not responding, timed out
[24552.102679] nfs: server 10.240.32.204 not responding, timed out
[24567.974667] nfs: server 10.240.32.204 not responding, timed out
[24576.166700] nfs: server 10.240.32.204 not responding, timed out
[24583.014698] nfs: server 10.240.32.204 not responding, timed out
[24592.038712] nfs: server 10.240.32.204 not responding, timed out
[24601.062750] nfs: server 10.240.32.204 not responding, timed out
[24706.343127] nfs: server 10.240.32.204 not responding, timed out
[24711.527033] nfs: server 10.240.32.204 not responding, timed out
[24717.543054] nfs: server 10.240.32.204 not responding, timed out
[24724.391046] nfs: server 10.240.32.204 not responding, timed out
[24733.415085] nfs: server 10.240.32.204 not responding, timed out
[24739.431113] nfs: server 10.240.32.204 not responding, timed out
[24754.471124] nfs: server 10.240.32.204 not responding, timed out
[24763.495177] nfs: server 10.240.32.204 not responding, timed out
[24769.511188] nfs: server 10.240.32.204 not responding, timed out
[24777.703199] nfs: server 10.240.32.204 not responding, timed out
[24787.559220] nfs: server 10.240.32.204 not responding, timed out
[24813.223285] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[24851.111490] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[24851.113197] LNetError: Skipped 10 previous similar messages
[24888.999493] nfs: server 10.240.32.204 not responding, timed out
[24898.855515] nfs: server 10.240.32.204 not responding, timed out
[24904.871543] nfs: server 10.240.32.204 not responding, timed out
[24910.887529] nfs: server 10.240.32.204 not responding, timed out
[24919.079578] nfs: server 10.240.32.204 not responding, timed out
[24925.927585] nfs: server 10.240.32.204 not responding, timed out
[24940.135646] nfs: server 10.240.32.204 not responding, timed out
[24949.991621] nfs: server 10.240.32.204 not responding, timed out
[24956.007645] nfs: server 10.240.32.204 not responding, timed out
[24965.031720] nfs: server 10.240.32.204 not responding, timed out
[24973.223685] nfs: server 10.240.32.204 not responding, timed out
[25070.311935] nfs: server 10.240.32.204 not responding, timed out
[25076.327959] nfs: server 10.240.32.204 not responding, timed out
[25085.351987] nfs: server 10.240.32.204 not responding, timed out
[25091.368074] nfs: server 10.240.32.204 not responding, timed out
[25096.552013] nfs: server 10.240.32.204 not responding, timed out
[25106.408080] nfs: server 10.240.32.204 not responding, timed out
[25111.592057] nfs: server 10.240.32.204 not responding, timed out
[25113.256055] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[25127.464098] nfs: server 10.240.32.204 not responding, timed out
[25136.488115] nfs: server 10.240.32.204 not responding, timed out
[25141.672204] nfs: server 10.240.32.204 not responding, timed out
[25150.696151] nfs: server 10.240.32.204 not responding, timed out
[25160.552170] nfs: server 10.240.32.204 not responding, timed out
[25250.792401] nfs: server 10.240.32.204 not responding, timed out
[25256.808407] nfs: server 10.240.32.204 not responding, timed out
[25262.824427] nfs: server 10.240.32.204 not responding, timed out
[25271.848452] nfs: server 10.240.32.204 not responding, timed out
[25283.048475] nfs: server 10.240.32.204 not responding, timed out
[25292.072554] nfs: server 10.240.32.204 not responding, timed out
[25298.920513] nfs: server 10.240.32.204 not responding, timed out
[25313.960584] nfs: server 10.240.32.204 not responding, timed out
[25322.152568] nfs: server 10.240.32.204 not responding, timed out
[25329.000577] nfs: server 10.240.32.204 not responding, timed out
[25338.024620] nfs: server 10.240.32.204 not responding, timed out
[25346.216630] nfs: server 10.240.32.204 not responding, timed out
[25413.288814] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[25431.272853] nfs: server 10.240.32.204 not responding, timed out
[25437.288869] nfs: server 10.240.32.204 not responding, timed out
[25443.304916] nfs: server 10.240.32.204 not responding, timed out
[25448.488903] nfs: server 10.240.32.204 not responding, timed out
[25453.225006] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[25453.226624] LNetError: Skipped 9 previous similar messages
[25458.344973] nfs: server 10.240.32.204 not responding, timed out
[25469.544940] nfs: server 10.240.32.204 not responding, timed out
[25479.401038] nfs: server 10.240.32.204 not responding, timed out
[25485.417003] nfs: server 10.240.32.204 not responding, timed out
[25499.625031] nfs: server 10.240.32.204 not responding, timed out
[25508.649078] nfs: server 10.240.32.204 not responding, timed out
[25515.497077] nfs: server 10.240.32.204 not responding, timed out
[25524.521089] nfs: server 10.240.32.204 not responding, timed out
[25533.545134] nfs: server 10.240.32.204 not responding, timed out
[25611.753338] nfs: server 10.240.32.204 not responding, timed out
[25616.937313] nfs: server 10.240.32.204 not responding, timed out
[25623.785351] nfs: server 10.240.32.204 not responding, timed out
[25629.801345] nfs: server 10.240.32.204 not responding, timed out
[25635.817391] nfs: server 10.240.32.204 not responding, timed out
[25644.841387] nfs: server 10.240.32.204 not responding, timed out
[25656.873442] nfs: server 10.240.32.204 not responding, timed out
[25665.897443] nfs: server 10.240.32.204 not responding, timed out
[25671.081482] nfs: server 10.240.32.204 not responding, timed out
[25686.953503] nfs: server 10.240.32.204 not responding, timed out
[25695.977524] nfs: server 10.240.32.204 not responding, timed out
[25701.161550] nfs: server 10.240.32.204 not responding, timed out
[25711.017575] nfs: server 10.240.32.204 not responding, timed out
[25713.321581] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[25720.041594] nfs: server 10.240.32.204 not responding, timed out
[25794.409769] nfs: server 10.240.32.204 not responding, timed out
[25798.249786] nfs: server 10.240.32.204 not responding, timed out
[25803.433838] nfs: server 10.240.32.204 not responding, timed out
[25816.297821] nfs: server 10.240.32.204 not responding, timed out
[25816.299024] nfs: server 10.240.32.204 not responding, timed out
[25822.313848] nfs: server 10.240.32.204 not responding, timed out
[25831.337877] nfs: server 10.240.32.204 not responding, timed out
[25842.537888] nfs: server 10.240.32.204 not responding, timed out
[25852.393918] nfs: server 10.240.32.204 not responding, timed out
[25858.409923] nfs: server 10.240.32.204 not responding, timed out
[25872.617989] nfs: server 10.240.32.204 not responding, timed out
[25882.474023] nfs: server 10.240.32.204 not responding, timed out
[25888.490017] nfs: server 10.240.32.204 not responding, timed out
[25896.682030] nfs: server 10.240.32.204 not responding, timed out
[25906.538119] nfs: server 10.240.32.204 not responding, timed out
[25975.722278] nfs: server 10.240.32.204 not responding, timed out
[25983.914256] nfs: server 10.240.32.204 not responding, timed out
[25990.762265] nfs: server 10.240.32.204 not responding, timed out
[26002.794316] nfs: server 10.240.32.204 not responding, timed out
[26007.978332] nfs: server 10.240.32.204 not responding, timed out
[26013.354340] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[26017.834342] nfs: server 10.240.32.204 not responding, timed out
[26029.866376] nfs: server 10.240.32.204 not responding, timed out
[26038.890409] nfs: server 10.240.32.204 not responding, timed out
[26044.906418] nfs: server 10.240.32.204 not responding, timed out
[26055.338538] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[26055.340252] LNetError: Skipped 10 previous similar messages
[26059.114431] nfs: server 10.240.32.204 not responding, timed out
[26068.970475] nfs: server 10.240.32.204 not responding, timed out
[26074.986485] nfs: server 10.240.32.204 not responding, timed out
[26084.010505] nfs: server 10.240.32.204 not responding, timed out
[26093.034579] nfs: server 10.240.32.204 not responding, timed out
[26158.378719] nfs: server 10.240.32.204 not responding, timed out
[26162.218710] nfs: server 10.240.32.204 not responding, timed out
[26171.242725] nfs: server 10.240.32.204 not responding, timed out
[26177.258755] nfs: server 10.240.32.204 not responding, timed out
[26188.458783] nfs: server 10.240.32.204 not responding, timed out
[26195.306787] nfs: server 10.240.32.204 not responding, timed out
[26204.330835] nfs: server 10.240.32.204 not responding, timed out
[26215.530832] nfs: server 10.240.32.204 not responding, timed out
[26225.386862] nfs: server 10.240.32.204 not responding, timed out
[26230.570868] nfs: server 10.240.32.204 not responding, timed out
[26246.443008] nfs: server 10.240.32.204 not responding, timed out
[26254.634943] nfs: server 10.240.32.204 not responding, timed out
[26260.650948] nfs: server 10.240.32.204 not responding, timed out
[26279.530991] nfs: server 10.240.32.204 not responding, timed out
[26279.532273] nfs: server 10.240.32.204 not responding, timed out
[26313.387079] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[26339.691173] nfs: server 10.240.32.204 not responding, timed out
[26347.883185] nfs: server 10.240.32.204 not responding, timed out
[26357.739223] nfs: server 10.240.32.204 not responding, timed out
[26363.755208] nfs: server 10.240.32.204 not responding, timed out
[26375.787234] nfs: server 10.240.32.204 not responding, timed out
[26381.803284] nfs: server 10.240.32.204 not responding, timed out
[26390.827282] nfs: server 10.240.32.204 not responding, timed out
[26402.859301] nfs: server 10.240.32.204 not responding, timed out
[26411.883345] nfs: server 10.240.32.204 not responding, timed out
[26417.899363] nfs: server 10.240.32.204 not responding, timed out
[26432.107391] nfs: server 10.240.32.204 not responding, timed out
[26441.963473] nfs: server 10.240.32.204 not responding, timed out
[26447.979435] nfs: server 10.240.32.204 not responding, timed out
[26465.195464] nfs: server 10.240.32.204 not responding, timed out
[26522.347604] nfs: server 10.240.32.204 not responding, timed out
[26526.187716] nfs: server 10.240.32.204 not responding, timed out
[26535.211644] nfs: server 10.240.32.204 not responding, timed out
[26543.403657] nfs: server 10.240.32.204 not responding, timed out
[26550.251714] nfs: server 10.240.32.204 not responding, timed out
[26562.283713] nfs: server 10.240.32.204 not responding, timed out
[26568.299727] nfs: server 10.240.32.204 not responding, timed out
[26577.323741] nfs: server 10.240.32.204 not responding, timed out
[26589.355783] nfs: server 10.240.32.204 not responding, timed out
[26598.379844] nfs: server 10.240.32.204 not responding, timed out
[26604.395810] nfs: server 10.240.32.204 not responding, timed out
[26613.419862] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[26619.435878] nfs: server 10.240.32.204 not responding, timed out
[26628.459895] nfs: server 10.240.32.204 not responding, timed out
[26633.643927] nfs: server 10.240.32.204 not responding, timed out
[26651.691926] nfs: server 10.240.32.204 not responding, timed out
[26658.476037] LNetError: Unexpected error -2 connecting to 10.240.45.178@tcp at host 10.240.45.178:7988
[26658.477720] LNetError: Skipped 10 previous similar messages
[26703.660098] nfs: server 10.240.32.204 not responding, timed out
[26712.684108] nfs: server 10.240.32.204 not responding, timed out
[26721.708111] nfs: server 10.240.32.204 not responding, timed out
[26729.900158] nfs: server 10.240.32.204 not responding, timed out
[26736.748147] nfs: server 10.240.32.204 not responding, timed out
[26748.780185] nfs: server 10.240.32.204 not responding, timed out
[26763.820256] nfs: server 10.240.32.204 not responding, timed out
[26763.821435] nfs: server 10.240.32.204 not responding, timed out
[26775.852290] nfs: server 10.240.32.204 not responding, timed out
[26784.876273] nfs: server 10.240.32.204 not responding, timed out
[26790.060320] nfs: server 10.240.32.204 not responding, timed out
[26805.932349] nfs: server 10.240.32.204 not responding, timed out
[26820.140361] nfs: server 10.240.32.204 not responding, timed out
[26838.188398] nfs: server 10.240.32.204 not responding, timed out
[26884.140525] nfs: server 10.240.32.204 not responding, timed out
[26890.156566] nfs: server 10.240.32.204 not responding, timed out
[26898.348557] nfs: server 10.240.32.204 not responding, timed out
[26908.204674] nfs: server 10.240.32.204 not responding, timed out
[26913.452629] LNet: 2 peer NIs in recovery (showing 2): 10.240.45.178@tcp, 10.240.44.153@tcp
[26917.228608] nfs: server 10.240.32.204 not responding, timed out
[26923.244643] nfs: server 10.240.32.204 not responding, timed out
[26935.276664] nfs: server 10.240.32.204 not responding, timed out
[26949.484713] nfs: server 10.240.32.204 not responding, timed out
[26962.348816] nfs: server 10.240.32.204 not responding, timed out
[26970.540741] nfs: server 10.240.32.204 not responding, timed out
[26977.388822] nfs: server 10.240.32.204 not responding, timed out
[26992.428829] nfs: server 10.240.32.204 not responding, timed out
[27007.468875] nfs: server 10.240.32.204 not responding, timed out
[27025.516908] nfs: server 10.240.32.204 not responding, timed out
[27064.620996] nfs: server 10.240.32.204 not responding, timed out
[27070.637059] nfs: server 10.240.32.204 not responding, timed out
[27075.821004] nfs: server 10.240.32.204 not responding, timed out
[27084.845032] nfs: server 10.240.32.204 not responding, timed out
[27094.701120] nfs: server 10.240.32.204 not responding, timed out
[27103.725101] nfs: server 10.240.32.204 not responding, timed out
[27109.741163] nfs: server 10.240.32.204 not responding, timed out
[27121.773124] nfs: server 10.240.32.204 not responding, timed out
[27135.981247] nfs: server 10.240.32.204 not responding, timed out
[27148.845212] nfs: server 10.240.32.204 not responding, timed out
[27157.037212] nfs: server 10.240.32.204 not responding, timed out
<ConMan> Console [trevis-106vm14] disconnected from <trevis-106:6013>.
<ConMan> Console [trevis-106vm14] connected to <trevis-106:6013>.
.......... ok
net0: 10.240.44.153/255.255.240.0 gw 10.240.32.1
Next server: 10.240.32.5
Filename: /pxelinux.0
tftp://10.240.32.5//pxelinux.0... ok
pxelinux.0 : 26759 bytes [PXE-NBP]
[1S
PXELINUX 4.05 0x5bd8f633 Copyright (C) 1994-2011 H. Peter Anvin et al[1S
!PXE entry point found (we hope) at 9C2C:0160 via plan A[1S
UNDI code segment at 9C2C len 0802[1S
UNDI data segment at 9CB0 len 2CE0[1S
Getting cached packet 01 02 03[1S
My IP address seems to be 0AF02C99 10.240.44.153[1S
ip=10.240.44.153:10.240.32.5:10.240.32.1:255.255.240.0[1S
BOOTIF=01-52-54-00-3e-d9-7f[1S
SYSUUID=10d44b74-1c64-444b-89e6-a7d1cc9fda86[1S
TFTP prefix: /[1S
Trying to load: pxelinux.cfg/10d44b74-1c64-444b-89e6-a7d1cc9fda86
Trying to load: pxelinux.cfg/01-52-54-00-3e-d9-7f ok[1S
Booting from local disk...[1S
No more network devices
Press Ctrl-B for the iPXE command line...
Booting from Hard Disk...
Loading Linux 5.15.0-94-generic ...
Loading initial ramdisk ...
[ 0.000000] Linux version 5.15.0-94-generic (buildd@lcy02-amd64-096) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 (Ubuntu 5.15.0-94.104-generic 5.15.136)
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=a72e3805-c7e8-4bb3-840b-92bd865f2f09 ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 slab_nomerge slub_nomerge crashkernel=512M-:296M
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Hygon HygonGenuine
[ 0.000000] Centaur CentaurHauls
[ 0.000000] zhaoxin Shanghai
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfff9fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000bfffa000-0x00000000bfffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.4 present.
[ 0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr ba001001, primary cpu clock
[ 0.000004] kvm-clock: using sched offset of 21411896883 cycles
[ 0.000011] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000024] tsc: Detected 2294.686 MHz processor
[ 0.000298] last_pfn = 0xbfffa max_arch_pfn = 0x400000000
[ 0.000347] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.012220] found SMP MP-table at [mem 0x000f6390-0x000f639f]
[ 0.012264] Using GB pages for direct mapping
[ 0.012522] RAMDISK: [mem 0x2b8bf000-0x31c56fff]
[ 0.012558] ACPI: Early table checksum verification disabled
[ 0.012589] ACPI: RSDP 0x00000000000F61C0 000014 (v00 BOCHS )
[ 0.012601] ACPI: RSDT 0x00000000BFFFFB40 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.012619] ACPI: FACP 0x00000000BFFFF177 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.012636] ACPI: DSDT 0x00000000BFFFE040 001137 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.012646] ACPI: FACS 0x00000000BFFFE000 000040
[ 0.012651] ACPI: SSDT 0x00000000BFFFF1EB 0008D5 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.012656] ACPI: APIC 0x00000000BFFFFAC0 000080 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.012660] ACPI: Reserving FACP table memory at [mem 0xbffff177-0xbffff1ea]
[ 0.012662] ACPI: Reserving DSDT table memory at [mem 0xbfffe040-0xbffff176]
[ 0.012664] ACPI: Reserving FACS table memory at [mem 0xbfffe000-0xbfffe03f]
[ 0.012666] ACPI: Reserving SSDT table memory at [mem 0xbffff1eb-0xbffffabf]
[ 0.012667] ACPI: Reserving APIC table memory at [mem 0xbffffac0-0xbffffb3f]
[ 0.013182] No NUMA configuration found
[ 0.013185] Faking a node at [mem 0x0000000000000000-0x00000000bfff9fff]
[ 0.013203] NODE_DATA(0) allocated [mem 0xbffd0000-0xbfff9fff]
[ 0.013731] Reserving 296MB of memory at 2624MB for crashkernel (System RAM: 3071MB)
[ 0.013777] Zone ranges:
[ 0.013784] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.013790] DMA32 [mem 0x0000000001000000-0x00000000bfff9fff]
[ 0.013798] Normal empty
[ 0.013800] Device empty
[ 0.013801] Movable zone start for each node
[ 0.013807] Early memory node ranges
[ 0.013808] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.013810] node 0: [mem 0x0000000000100000-0x00000000bfff9fff]
[ 0.013812] Initmem setup node 0 [mem 0x0000000000001000-0x00000000bfff9fff]
[ 0.013840] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.014117] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.051967] On node 0, zone DMA32: 6 pages in unavailable ranges
[ 0.052526] ACPI: PM-Timer IO Port: 0x608
[ 0.052556] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.052607] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.052613] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.052615] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.052622] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.052630] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.052631] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.052637] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.052644] TSC deadline timer available
[ 0.052646] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.052685] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.052688] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[ 0.052690] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[ 0.052691] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[ 0.052696] [mem 0xc0000000-0xfeffbfff] available for PCI devices
[ 0.052698] Booting paravirtualized kernel on KVM
[ 0.052715] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.052731] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.055100] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u1048576
[ 0.055141] kvm-guest: stealtime: cpu 0, msr bcc33080
[ 0.055145] kvm-guest: PV spinlocks enabled
[ 0.055148] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)
[ 0.055167] Built 1 zonelists, mobility grouping on. Total pages: 773882
[ 0.055170] Policy zone: DMA32
[ 0.055172] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=a72e3805-c7e8-4bb3-840b-92bd865f2f09 ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 slab_nomerge slub_nomerge crashkernel=512M-:296M
[ 0.055554] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic biosdevname=0", will be passed to user space.
[ 0.059994] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.062210] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.062290] mem auto-init: stack:off, heap alloc:on, heap free:off
[ 0.070497] Memory: 2625904K/3145312K available (16393K kernel code, 4394K rwdata, 10876K rodata, 3356K init, 18716K bss, 519148K reserved, 0K cma-reserved)
[ 0.070931] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.070968] Kernel/User page tables isolation: enabled
[ 0.071060] ftrace: allocating 50666 entries in 198 pages
[ 0.101925] ftrace: allocated 198 pages with 4 groups
[ 0.102300] rcu: Hierarchical RCU implementation.
[ 0.102302] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=2.
[ 0.102304] Rude variant of Tasks RCU enabled.
[ 0.102305] Tracing variant of Tasks RCU enabled.
[ 0.102309] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.102311] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.107803] NR_IRQS: 524544, nr_irqs: 440, preallocated irqs: 16
[ 0.108012] random: crng init done
[ 0.108987] Console: colour VGA+ 80x25
[ 0.119562] printk: console [tty0] enabled
[ 0.222167] printk: console [ttyS0] enabled
[ 0.222944] ACPI: Core revision 20210730
[ 0.223768] APIC: Switch to symmetric I/O mode setup
[ 0.224954] x2apic enabled
[ 0.225831] Switched APIC routing to physical x2apic.
[ 0.228157] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.229362] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x211399552f8, max_idle_ns: 440795292447 ns
[ 0.231318] Calibrating delay loop (skipped) preset value.. 4589.37 BogoMIPS (lpj=9178744)
[ 0.235436] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.236477] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.237608] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.239316] Spectre V2 : Mitigation: Retpolines
[ 0.240205] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.241745] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.243314] Speculative Store Bypass: Vulnerable
[ 0.244236] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[ 0.245484] TAA: Vulnerable: Clear CPU buffers attempted, no microcode
[ 0.247314] MMIO Stale Data: Unknown: No mitigations
[ 0.248284] SRBDS: Unknown: Dependent on hypervisor status
[ 0.249363] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.250832] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.251314] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.252534] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.253714] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.277118] Freeing SMP alternatives memory: 44K
[ 0.278107] pid_max: default: 32768 minimum: 301
[ 0.279067] LSM: Security Framework initializing
[ 0.279333] landlock: Up and running.
[ 0.280094] Yama: becoming mindful.
[ 0.280891] AppArmor: AppArmor initialized
[ 0.281819] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.283373] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.285462] smpboot: CPU0: Intel Core Processor (Broadwell) (family: 0x6, model: 0x3d, stepping: 0x2)
[ 0.287313] Performance Events: unsupported p6 CPU model 61 no PMU driver, software events only.
[ 0.287313] signal: max sigframe size: 1776
[ 0.287360] rcu: Hierarchical SRCU implementation.
[ 0.289013] NMI watchdog: Perf NMI watchdog permanently disabled
[ 0.290291] smp: Bringing up secondary CPUs ...
[ 0.291477] x86: Booting SMP configuration:
[ 0.292333] .... node #0, CPUs: #1
[ 0.123310] kvm-clock: cpu 1, msr ba001041, secondary cpu clock
[ 0.123310] smpboot: CPU 1 Converting physical 0 to logical die 1
[ 0.295341] kvm-guest: stealtime: cpu 1, msr bcd33080
[ 0.296257] smp: Brought up 1 node, 2 CPUs
[ 0.296257] smpboot: Max logical packages: 2
[ 0.296842] smpboot: Total of 2 processors activated (9178.74 BogoMIPS)
[ 0.299436] devtmpfs: initialized
[ 0.300036] x86/mm: Memory block size: 128MB
[ 0.301282] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.303345] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.304532] pinctrl core: initialized pinctrl subsystem
[ 0.305704] PM: RTC time: 21:15:43, date: 2024-12-16
[ 0.306823] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.307923] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[ 0.309567] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[ 0.311304] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[ 0.311328] audit: initializing netlink subsys (disabled)
[ 0.312303] audit: type=2000 audit(1734383746.766:1): state=initialized audit_enabled=0 res=1
[ 0.312303] thermal_sys: Registered thermal governor 'fair_share'
[ 0.312787] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.313817] thermal_sys: Registered thermal governor 'step_wise'
[ 0.315318] thermal_sys: Registered thermal governor 'user_space'
[ 0.316332] thermal_sys: Registered thermal governor 'power_allocator'
[ 0.317359] EISA bus registered
[ 0.319034] cpuidle: using governor ladder
[ 0.319320] cpuidle: using governor menu
[ 0.320152] ACPI: bus type PCI registered
[ 0.320878] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 0.322189] PCI: Using configuration type 1 for base access
[ 0.324770] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.326297] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.327324] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.328516] ACPI: Added _OSI(Module Device)
[ 0.331319] ACPI: Added _OSI(Processor Device)
[ 0.332097] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.332909] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.333824] ACPI: Added _OSI(Linux-Dell-Video)
[ 0.334603] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[ 0.335316] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[ 0.337734] ACPI: 2 ACPI AML tables successfully acquired and loaded
[ 0.339690] ACPI: Interpreter enabled
[ 0.340397] ACPI: PM: (supports S0 S5)
[ 0.341069] ACPI: Using IOAPIC for interrupt routing
[ 0.341938] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.343324] PCI: Using E820 reservations for host bridge windows
[ 0.344603] ACPI: Enabled 16 GPEs in block 00 to 0F
[ 0.348635] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.349718] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[ 0.351059] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 0.351702] acpiphp: Slot [3] registered
[ 0.352435] acpiphp: Slot [4] registered
[ 0.353153] acpiphp: Slot [5] registered
[ 0.353875] acpiphp: Slot [6] registered
[ 0.354597] acpiphp: Slot [7] registered
[ 0.355320] acpiphp: Slot [8] registered
[ 0.356049] acpiphp: Slot [9] registered
[ 0.356774] acpiphp: Slot [10] registered
[ 0.357516] acpiphp: Slot [11] registered
[ 0.358246] acpiphp: Slot [12] registered
[ 0.358983] acpiphp: Slot [13] registered
[ 0.359345] acpiphp: Slot [14] registered
[ 0.360092] acpiphp: Slot [15] registered
[ 0.360826] acpiphp: Slot [16] registered
[ 0.361562] acpiphp: Slot [17] registered
[ 0.362298] acpiphp: Slot [18] registered
[ 0.363030] acpiphp: Slot [19] registered
[ 0.363344] acpiphp: Slot [20] registered
[ 0.364085] acpiphp: Slot [21] registered
[ 0.364827] acpiphp: Slot [22] registered
[ 0.365556] acpiphp: Slot [23] registered
[ 0.366290] acpiphp: Slot [24] registered
[ 0.367034] acpiphp: Slot [25] registered
[ 0.367345] acpiphp: Slot [26] registered
[ 0.368088] acpiphp: Slot [27] registered
[ 0.368825] acpiphp: Slot [28] registered
[ 0.369564] acpiphp: Slot [29] registered
[ 0.370299] acpiphp: Slot [30] registered
[ 0.371034] acpiphp: Slot [31] registered
[ 0.371332] PCI host bridge to bus 0000:00
[ 0.372064] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.373002] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.374126] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.375251] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 0.375316] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[ 0.376666] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[ 0.378555] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[ 0.380115] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[ 0.386663] pci 0000:00:01.1: reg 0x20: [io 0xc0a0-0xc0af]
[ 0.389580] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.390774] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.391316] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.392492] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.393874] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
[ 0.399893] pci 0000:00:01.2: reg 0x20: [io 0xc040-0xc05f]
[ 0.403478] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[ 0.405086] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.406314] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.408125] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
[ 0.411317] pci 0000:00:03.0: reg 0x10: [io 0xc060-0xc07f]
[ 0.414422] pci 0000:00:03.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff]
[ 0.425451] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
[ 0.427393] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000
[ 0.430673] pci 0000:00:04.0: reg 0x10: [io 0xc000-0xc03f]
[ 0.433462] pci 0000:00:04.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff]
[ 0.445176] pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00
[ 0.447317] pci 0000:00:05.0: reg 0x10: [io 0xc080-0xc09f]
[ 0.460686] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[ 0.461859] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[ 0.463020] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
[ 0.463442] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[ 0.464530] ACPI: PCI: Interrupt link LNKS configured for IRQ 9
[ 0.465886] iommu: Default domain type: Translated
[ 0.465886] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.467588] SCSI subsystem initialized
[ 0.468328] vgaarb: loaded
[ 0.468328] ACPI: bus type USB registered
[ 0.468658] usbcore: registered new interface driver usbfs
[ 0.469614] usbcore: registered new interface driver hub
[ 0.471324] usbcore: registered new device driver usb
[ 0.472234] pps_core: LinuxPPS API ver. 1 registered
[ 0.473085] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.474579] PTP clock support registered
[ 0.475344] EDAC MC: Ver: 3.0.0
[ 0.476215] NetLabel: Initializing
[ 0.476215] NetLabel: domain hash size = 128
[ 0.476954] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO
[ 0.477944] NetLabel: unlabeled traffic allowed by default
[ 0.479346] PCI: Using ACPI for IRQ routing
[ 0.483324] clocksource: Switched to clocksource kvm-clock
[ 0.509916] VFS: Disk quotas dquot_6.6.0
[ 0.510726] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.512088] AppArmor: AppArmor Filesystem Enabled
[ 0.512944] pnp: PnP ACPI init
[ 0.514060] pnp: PnP ACPI: found 5 devices
[ 0.521804] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.523362] NET: Registered PF_INET protocol family
[ 0.524706] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.528730] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.530386] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.531912] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.533703] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[ 0.534949] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.536322] MPTCP token hash table entries: 4096 (order: 4, 98304 bytes, linear)
[ 0.537647] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.538838] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.540075] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.541050] NET: Registered PF_XDP protocol family
[ 0.541895] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.542938] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.543979] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[ 0.545126] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
[ 0.546320] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.547309] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.548330] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.549714] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[ 0.551289] PCI: CLS 0 bytes, default 64
[ 0.552171] Trying to unpack rootfs image as initramfs...
[ 0.559448] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x211399552f8, max_idle_ns: 440795292447 ns
[ 0.567920] Initialise system trusted keyrings
[ 0.568763] Key type blacklist registered
[ 0.575358] workingset: timestamp_bits=36 max_order=20 bucket_order=0
[ 0.578484] zbud: loaded
[ 0.579573] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.580867] fuse: init (API version 7.34)
[ 0.581813] integrity: Platform Keyring initialized
[ 0.592769] Key type asymmetric registered
[ 0.593540] Asymmetric key parser 'x509' registered
[ 0.594424] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
[ 0.595758] io scheduler mq-deadline registered
[ 0.596991] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 0.598260] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.599572] ACPI: button: Power Button [PWRF]
[ 0.601002] ACPI: \_SB_.LNKC: Enabled at IRQ 10
[ 0.601897] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
[ 0.603367] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[ 0.605386] ACPI: \_SB_.LNKA: Enabled at IRQ 10
[ 0.606246] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[ 0.608457] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[ 0.632299] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 0.635589] Linux agpgart interface v0.103
[ 0.638868] loop: module loaded
[ 0.641058] scsi host0: ata_piix
[ 0.641866] scsi host1: ata_piix
[ 0.642525] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
[ 0.643676] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
[ 0.644971] tun: Universal TUN/TAP device driver, 1.6
[ 0.646193] PPP generic driver version 2.4.2
[ 0.647008] VFIO - User Level meta-driver version: 0.3
[ 0.648018] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.649396] ehci-pci: EHCI PCI platform driver
[ 0.650210] ehci-platform: EHCI generic platform driver
[ 0.651117] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 0.652179] ohci-pci: OHCI PCI platform driver
[ 0.652971] ohci-platform: OHCI generic platform driver
[ 0.653900] uhci_hcd: USB Universal Host Controller Interface driver
[ 0.656144] uhci_hcd 0000:00:01.2: UHCI Host Controller
[ 0.657084] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
[ 0.658337] uhci_hcd 0000:00:01.2: detected 2 ports
[ 0.659323] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
[ 0.660457] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 5.15
[ 0.661879] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 0.663132] usb usb1: Product: UHCI Host Controller
[ 0.664018] usb usb1: Manufacturer: Linux 5.15.0-94-generic uhci_hcd
[ 0.665112] usb usb1: SerialNumber: 0000:00:01.2
[ 0.666059] hub 1-0:1.0: USB hub found
[ 0.666782] hub 1-0:1.0: 2 ports detected
[ 0.667769] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[ 0.670039] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.670944] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.672001] mousedev: PS/2 mouse device common for all mice
[ 0.673088] rtc_cmos 00:00: RTC can wake from S4
[ 0.674632] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[ 0.676505] rtc_cmos 00:00: registered as rtc0
[ 0.677438] rtc_cmos 00:00: setting system clock to 2024-12-16T21:15:44 UTC (1734383744)
[ 0.678892] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
[ 0.679960] i2c_dev: i2c /dev entries driver
[ 0.680779] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[ 0.682795] device-mapper: uevent: version 1.0.3
[ 0.683700] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[ 0.685150] platform eisa.0: Probing EISA bus 0
[ 0.685983] platform eisa.0: EISA: Cannot allocate resource for mainboard
[ 0.687143] platform eisa.0: Cannot allocate resource for EISA slot 1
[ 0.688256] platform eisa.0: Cannot allocate resource for EISA slot 2
[ 0.689361] platform eisa.0: Cannot allocate resource for EISA slot 3
[ 0.690461] platform eisa.0: Cannot allocate resource for EISA slot 4
[ 0.691560] platform eisa.0: Cannot allocate resource for EISA slot 5
[ 0.692661] platform eisa.0: Cannot allocate resource for EISA slot 6
[ 0.693786] platform eisa.0: Cannot allocate resource for EISA slot 7
[ 0.694857] platform eisa.0: Cannot allocate resource for EISA slot 8
[ 0.695957] platform eisa.0: EISA: Detected 0 cards
[ 0.696834] intel_pstate: CPU model not supported
[ 0.697720] ledtrig-cpu: registered to indicate activity on CPUs
[ 0.698872] drop_monitor: Initializing network drop monitor service
[ 0.700164] NET: Registered PF_INET6 protocol family
[ 1.606521] Freeing initrd memory: 101984K
[ 1.613116] Segment Routing with IPv6
[ 1.613880] In-situ OAM (IOAM) with IPv6
[ 1.614629] NET: Registered PF_PACKET protocol family
[ 1.615593] Key type dns_resolver registered
[ 1.616668] IPI shorthand broadcast: enabled
[ 1.617472] sched_clock: Marking stable (1497197409, 119310520)->(1651785548, -35277619)
[ 1.619012] registered taskstats version 1
[ 1.619854] Loading compiled-in X.509 certificates
[ 1.621466] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[ 1.623704] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969'
[ 1.625920] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19'
[ 1.627552] blacklist: Loading compiled-in revocation X.509 certificates
[ 1.628696] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing: 61482aa2830d0ab2ad5af10b7250da9033ddcef0'
[ 1.630295] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2017): 242ade75ac4a15e50d50c84b0d45ff3eae707a03'
[ 1.631996] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (ESM 2018): 365188c1d374d6b07c3c8f240f8ef722433d6a8b'
[ 1.633738] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2019): c0746fd6c5da3ae827864651ad66ae47fe24b3e8'
[ 1.635446] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d'
[ 1.637167] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v2): 4cf046892d6fd3c9a5b03f98d845f90851dc6a8c'
[ 1.638924] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v3): 100437bb6de6e469b581e61cd66bce3ef4ed53af'
[ 1.640680] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9'
[ 1.642784] zswap: loaded using pool lzo/zbud
[ 1.643765] Key type .fscrypt registered
[ 1.644486] Key type fscrypt-provisioning registered
[ 1.648054] Key type encrypted registered
[ 1.648807] AppArmor: AppArmor sha1 policy hashing enabled
[ 1.649747] ima: No TPM chip found, activating TPM-bypass!
[ 1.650689] Loading compiled-in module X.509 certificates
[ 1.652200] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[ 1.653809] ima: Allocated hash algorithm: sha1
[ 1.654605] ima: No architecture policies found
[ 1.655403] evm: Initialising EVM extended attributes:
[ 1.656281] evm: security.selinux
[ 1.656887] evm: security.SMACK64
[ 1.657492] evm: security.SMACK64EXEC
[ 1.658142] evm: security.SMACK64TRANSMUTE
[ 1.658863] evm: security.SMACK64MMAP
[ 1.659526] evm: security.apparmor
[ 1.660142] evm: security.ima
[ 1.660700] evm: security.capability
[ 1.661339] evm: HMAC attrs: 0x1
[ 1.662279] PM: Magic number: 4:189:298
[ 1.663254] RAS: Correctable Errors collector initialized.
[ 1.666049] Freeing unused decrypted memory: 2036K
[ 1.668356] Freeing unused kernel image (initmem) memory: 3356K
[ 1.679332] Write protecting the kernel read-only data: 30720k
[ 1.682394] Freeing unused kernel image (text/rodata gap) memory: 2036K
[ 1.684868] Freeing unused kernel image (rodata/data gap) memory: 1412K
[ 1.745976] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 1.747089] x86/mm: Checking user space page tables
[ 1.803943] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 1.805066] Run /init as init process
Loading, please wait...
Starting version 249.11-0ubuntu3.12
[ 1.944555] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
[ 1.948022] virtio_blk virtio1: [vda] 209715200 512-byte logical blocks (107 GB/100 GiB)
[ 1.949457] cryptd: max_cpu_qlen set to 1000
[ 1.951128] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
[ 1.952861] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
[ 1.959701] vda: vda1 vda4
[ 1.960771] AVX2 version of gcm_enc/dec engaged.
[ 1.961650] AES CTR mode by8 optimization enabled
[ 1.963873] FDC 0 is a S82078B
Begin: Loading essential drivers ... [ 3.227321] raid6: avx2x4 gen() 26957 MB/s
[ 3.295321] raid6: avx2x4 xor() 9230 MB/s
[ 3.363320] raid6: avx2x2 gen() 25660 MB/s
[ 3.431321] raid6: avx2x2 xor() 15977 MB/s
[ 3.499320] raid6: avx2x1 gen() 23671 MB/s
[ 3.567320] raid6: avx2x1 xor() 14072 MB/s
[ 3.635320] raid6: sse2x4 gen() 14430 MB/s
[ 3.703319] raid6: sse2x4 xor() 7703 MB/s
[ 3.771320] raid6: sse2x2 gen() 13780 MB/s
[ 3.839319] raid6: sse2x2 xor() 9214 MB/s
[ 3.907320] raid6: sse2x1 gen() 12155 MB/s
[ 3.979320] raid6: sse2x1 xor() 8195 MB/s
[ 3.980124] raid6: using algorithm avx2x4 gen() 26957 MB/s
[ 3.981064] raid6: .... xor() 9230 MB/s, rmw enabled
[ 3.981915] raid6: using avx2x2 recovery algorithm
[ 3.984272] xor: automatically using best checksumming function avx
[ 3.986364] async_tx: api initialized (async)
done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... [ 4.059008] Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes
Scanning for Btrfs filesystems
done.
Begin: Will now check root file system ... fsck from util-linux 2.37.2
[/usr/sbin/fsck.ext4 (1) -- /dev/vda1] fsck.ext4 -a -C0 /dev/vda1
/dev/vda1: recovering journal
/dev/vda1: clean, 149870/1310720 files, 2494899/5242880 blocks
done.
[ 4.369288] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[ 5.424120] systemd[1]: Inserted module 'autofs4'
[ 5.529634] systemd[1]: systemd 249.11-0ubuntu3.12 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 5.534574] systemd[1]: Detected virtualization kvm.
[ 5.535525] systemd[1]: Detected architecture x86-64.
Welcome to [1mUbuntu 22.04 LTS[0m!
[ 5.537644] systemd[1]: Hostname set to <trevis-106vm14.trevis.whamcloud.com>.
[ 6.654703] systemd[1]: Configuration file /run/systemd/system/netplan-ovs-cleanup.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
[ 7.196239] systemd[1]: Queued start job for default target Graphical Interface.
[ 7.198766] systemd[1]: Created slice Slice /system/modprobe.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/modprobe[0m.
[ 7.200900] systemd[1]: Created slice Slice /system/postfix.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/postfix[0m.
[ 7.202914] systemd[1]: Created slice Slice /system/serial-getty.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/serial-getty[0m.
[ 7.204884] systemd[1]: Created slice User and Session Slice.
[[0;32m OK [0m] Created slice [0;1;39mUser and Session Slice[0m.
[ 7.206620] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[[0;32m OK [0m] Started [0;1;39mForward Password R…uests to Wall Directory Watch[0m.
[ 7.208956] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[[0;32m OK [0m] Set up automount [0;1;39mArbitrary…s File System Automount Point[0m.
[ 7.211418] systemd[1]: Reached target Slice Units.
[[0;32m OK [0m] Reached target [0;1;39mSlice Units[0m.
[ 7.212893] systemd[1]: Reached target System Time Set.
[[0;32m OK [0m] Reached target [0;1;39mSystem Time Set[0m.
[ 7.214478] systemd[1]: Reached target Local Verity Protected Volumes.
[[0;32m OK [0m] Reached target [0;1;39mLocal Verity Protected Volumes[0m.
[ 7.216425] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[[0;32m OK [0m] Listening on [0;1;39mDevice-mapper event daemon FIFOs[0m.
[ 7.218427] systemd[1]: Listening on LVM2 poll daemon socket.
[[0;32m OK [0m] Listening on [0;1;39mLVM2 poll daemon socket[0m.
[ 7.220253] systemd[1]: Listening on multipathd control socket.
[[0;32m OK [0m] Listening on [0;1;39mmultipathd control socket[0m.
[ 7.225482] systemd[1]: Listening on RPCbind Server Activation Socket.
[[0;32m OK [0m] Listening on [0;1;39mRPCbind Server Activation Socket[0m.
[ 7.227761] systemd[1]: Listening on Syslog Socket.
[[0;32m OK [0m] Listening on [0;1;39mSyslog Socket[0m.
[ 7.229337] systemd[1]: Listening on fsck to fsckd communication Socket.
[[0;32m OK [0m] Listening on [0;1;39mfsck to fsckd communication Socket[0m.
[ 7.231347] systemd[1]: Listening on initctl Compatibility Named Pipe.
[[0;32m OK [0m] Listening on [0;1;39minitctl Compatibility Named Pipe[0m.
[ 7.233424] systemd[1]: Listening on Journal Audit Socket.
[[0;32m OK [0m] Listening on [0;1;39mJournal Audit Socket[0m.
[ 7.235138] systemd[1]: Listening on Journal Socket (/dev/log).
[[0;32m OK [0m] Listening on [0;1;39mJournal Socket (/dev/log)[0m.
[ 7.236982] systemd[1]: Listening on Journal Socket.
[[0;32m OK [0m] Listening on [0;1;39mJournal Socket[0m.
[ 7.238604] systemd[1]: Listening on Network Service Netlink Socket.
[[0;32m OK [0m] Listening on [0;1;39mNetwork Service Netlink Socket[0m.
[ 7.240616] systemd[1]: Listening on udev Control Socket.
[[0;32m OK [0m] Listening on [0;1;39mudev Control Socket[0m.
[ 7.242288] systemd[1]: Listening on udev Kernel Socket.
[[0;32m OK [0m] Listening on [0;1;39mudev Kernel Socket[0m.
[ 7.244656] systemd[1]: Mounting Huge Pages File System...
Mounting [0;1;39mHuge Pages File System[0m...
[ 7.247083] systemd[1]: Mounting POSIX Message Queue File System...
Mounting [0;1;39mPOSIX Message Queue File System[0m...
[ 7.249537] systemd[1]: Mounting NFSD configuration filesystem...
Mounting [0;1;39mNFSD configuration filesystem[0m...
[ 7.252144] systemd[1]: Mounting Kernel Debug File System...
Mounting [0;1;39mKernel Debug File System[0m...
[ 7.254604] systemd[1]: Mounting Kernel Trace File System...
Mounting [0;1;39mKernel Trace File System[0m...
[ 7.257889] systemd[1]: Starting Journal Service...
Starting [0;1;39mJournal Service[0m...
[ 7.259471] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS being skipped.
[ 7.262065] systemd[1]: Starting Set the console keyboard layout...
Starting [0;1;39mSet the console keyboard layout[0m...
[ 7.264940] systemd[1]: Starting Create List of Static Device Nodes...
Starting [0;1;39mCreate List of Static Device Nodes[0m...
[ 7.268948] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
Starting [0;1;39mMonitoring of LVM…meventd or progress polling[0m...
[ 7.277936] systemd[1]: Condition check resulted in LXD - agent being skipped.
[ 7.282756] systemd[1]: Starting Load Kernel Module configfs...
Starting [0;1;39mLoad Kernel Module configfs[0m...
[ 7.286922] systemd[1]: Starting Load Kernel Module drm...
Starting [0;1;39mLoad Kernel Module drm[0m...
[ 7.291089] systemd[1]: Starting Load Kernel Module efi_pstore...
Starting [0;1;39mLoad Kernel Module efi_pstore[0m...
[ 7.295470] systemd[1]: Starting Load Kernel Module fuse...
Starting [0;1;39mLoad Kernel Module fuse[0m...
[ 7.299189] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
[ 7.301327] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[ 7.329297] RPC: Registered named UNIX socket transport module.
[ 7.330392] RPC: Registered udp transport module.
[ 7.331228] RPC: Registered tcp transport module.
[ 7.332075] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 7.358713] systemd[1]: Starting Load Kernel Modules...
Starting [0;1;39mLoad Kernel Modules[0m...
[ 7.361364] systemd[1]: Starting Remount Root and Kernel File Systems...
Starting [0;1;39mRemount Root and Kernel File Systems[0m...
[ 7.366005] systemd[1]: Starting Coldplug All udev Devices...
Starting [0;1;39mColdplug All udev Devices[0m...
[ 7.369703] systemd[1]: Started Journal Service.
[[0;32m OK [0m] Started [0;1;39mJournal Service[0m.
[[0;32m OK [0m] Mounted [0;1;39mHuge Pages File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mPOSIX Message Queue File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Debug File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mKernel Trace File System[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate List of Static Device Nodes[0m.
[[0;32m OK [0m] Finished [0;1;39mMonitoring of LVM… dmeventd or progress polling[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module configfs[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module efi_pstore[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Module fuse[0m.
Mounting [0;1;39mFUSE Control File System[0m...
Mounting [0;1;39mKernel Configuration File System[0m...
[[0;32m OK [0m] Mounted [0;1;39mKernel Configuration File System[0m.
[[0;32m OK [0m] Mounted [0;1;39mFUSE Control File System[0m.
[[0;32m OK [0m] Finished [0;1;39mSet the console keyboard layout[0m.
[[0;32m OK [[ 7.545995] EXT4-fs (vda1): re-mounted. Opts: (null). Quota mode: none.
0m] Finished [0;1;39mLoad Kernel Module drm[0m.
[[0;32m OK [0m] Finished [0;1;39mRemount Root and Kernel File Systems[0m.
Activating swap [0;1;39m/swap.img[0m...
Starting [0;1;39mDevice-Mapper Multipath Device Controller[0m...
Starting [0;1;39mFlush Journal to Persistent Storage[0m...
Starting [0;1;39mLoad/Save Random Seed[0m...
[ 7.560358] systemd-journald[355]: Received client request to flush runtime journal.
Starting [0;1;39mCreate System Users[0m...
[[0;32m OK [0m] Finished [0;1;39mColdplug All udev Devices[0m.
[ 7.651227] alua: device handler registered
Starting [0;1;39mHelper to synchronize boot up for ifupdown[0m...
[ 7.654924] emc: device handler registered
[ 7.657172] rdac: device handler registered
[ 7.862146] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[[0;32m OK [0m] Mounted [0;1;39mNFSD configuration filesystem[0m.
[ 7.916671] IPMI message handler: version 39.2
[ 7.919247] ipmi device interface
[[0;32m OK [0m] Finished [0;1;39mLoad Kernel Modules[0m.
Starting [0;1;39mApply Kernel Variables[0m...
[ 7.956878] systemd-journald[355]: File /var/log/journal/2893385f3d7c4e94bd25f791e10d4326/system.journal corrupted or uncleanly shut down, renaming and replacing.
[ 7.956920] Adding 4194300k swap on /swap.img. Priority:-2 extents:4 across:5857276k FS
[[0;32m OK [0m] Activated swap [0;1;39m/swap.img[0m.
[[0;32m OK [0m] Finished [0;1;39mLoad/Save Random Seed[0m.
[[0;32m OK [0m] Reached target [0;1;39mSwaps[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate System Users[0m.
Starting [0;1;39mCreate Static Device Nodes in /dev[0m...
[[0;32m OK [0m] Finished [0;1;39mApply Kernel Variables[0m.
[[0;32m OK [0m] Finished [0;1;39mFlush Journal to Persistent Storage[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate Static Device Nodes in /dev[0m.
Starting [0;1;39mRule-based Manage…for Device Events and Files[0m...
[[0;32m OK [0m] Started [0;1;39mDevice-Mapper Multipath Device Controller[0m.
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Local File Systems[0m.
[[0;32m OK [0m] Reached target [0;1;39mLocal File Systems[0m.
Starting [0;1;39mLoad AppArmor profiles[0m...
Starting [0;1;39mSet console font and keymap[0m...
Starting [0;1;39mCreate final runt…dir for shutdown pivot root[0m...
Starting [0;1;39mTell Plymouth To Write Out Runtime Data[0m...
Starting [0;1;39mSet Up Additional Binary Formats[0m...
Starting [0;1;39mCreate Volatile Files and Directories[0m...
Starting [0;1;39mUncomplicated firewall[0m...
[[0;32m OK [0m] Finished [0;1;39mCreate final runt…e dir for shutdown pivot root[0m.
[[0;32m OK [0m] Finished [0;1;39mSet console font and keymap[0m.
Mounting [0;1;39mArbitrary Executable File Formats File System[0m...
[[0;32m OK [0m] Finished [0;1;39mUncomplicated firewall[0m.
[[0;32m OK [0m] Mounted [0;1;39mArbitrary Executable File Formats File System[0m.
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Network[0m.
[[0;32m OK [0m] Finished [0;1;39mSet Up Additional Binary Formats[0m.
[[0;32m OK [0m] Finished [0;1;39mTell Plymouth To Write Out Runtime Data[0m.
[[0;32m OK [0m] Finished [0;1;39mCreate Volatile Files and Directories[0m.
Mounting [0;1;39mRPC Pipe File System[0m...
Starting [0;1;39mRPC bind portmap service[0m...
Starting [0;1;39mRecord System Boot/Shutdown in UTMP[0m...
[[0;32m OK [0m] Mounted [0;1;39mRPC Pipe File System[0m.
[[0;32m OK [0m] Reached target [0;1;39mrpc_pipefs.target[0m.
Starting [0;1;39mpNFS block layout mapping daemon[0m...
Starting [0;1;39mNFSv4 ID-name mapping service[0m...
Starting [0;1;39mNFSv4 Client Tracking Daemon[0m...
[[0;32m OK [0m] Reached target [0;1;39mNFS client services[0m.
[[0;32m OK [0m] Started [0;1;39mpNFS block layout mapping daemon[0m.
[[0;32m OK [0m] Finished [0;1;39mRecord System Boot/Shutdown in UTMP[0m.
[[0;32m OK [0m] Started [0;1;39mRPC bind portmap service[0m.
[[0;32m OK [0m] Reached target [0;1;39mRPC Port Mapper[0m.
[[0;32m OK [0m] Started [0;1;39mNFSv4 Client Tracking Daemon[0m.
[[0;32m OK [0m] Started [0;1;39mNFSv4 ID-name mapping service[0m.
[ 8.754126] audit: type=1400 audit(1734383752.572:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=436 comm="apparmor_parser"
[ 8.756685] audit: type=1400 audit(1734383752.572:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=436 comm="apparmor_parser"
[ 8.761429] audit: type=1400 audit(1734383752.580:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lsb_release" pid=435 comm="apparmor_parser"
[ 8.770230] audit: type=1400 audit(1734383752.588:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=440 comm="apparmor_parser"
[ 8.772895] audit: type=1400 audit(1734383752.588:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=440 comm="apparmor_parser"
[ 8.775333] audit: type=1400 audit(1734383752.588:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_groff" pid=440 comm="apparmor_parser"
[[0;32m OK [0m] Started [0;1;39mRule-based Manager for Device Events and Files[0m.
[[0;32m OK [0m] Started [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[[0;32m OK [0m] Reached target [0;1;39mLocal Encrypted Volumes[0m.
Starting [0;1;39mNetwork Configuration[0m...
[ 8.851471] audit: type=1400 audit(1734383752.672:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=439 comm="apparmor_parser"
[ 8.854420] audit: type=1400 audit(1734383752.672:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=439 comm="apparmor_parser"
[ 8.857216] audit: type=1400 audit(1734383752.672:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=439 comm="apparmor_parser"
[ 8.860043] audit: type=1400 audit(1734383752.672:11): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/{,usr/}sbin/dhclient" pid=439 comm="apparmor_parser"
[[0;32m OK [0m] Finished [0;1;39mLoad AppArmor profiles[0m.
[[0;32m OK [0m] Reached target [0;1;39mSystem Initialization[0m.
[[0;32m OK [0m] Started [0;1;39mTrigger to poll fo…y enabled on GCP LTS non-pro)[0m.
[[0;32m OK [0m] Started [0;1;39mDaily Cleanup of Temporary Directories[0m.
[[0;32m OK [0m] Started [0;1;39mUbuntu Advantage Timer for running repeated jobs[0m.
[[0;32m OK [0m] Reached target [0;1;39mPath Units[0m.
[[0;32m OK [0m] Listening on [0;1;39mD-Bus System Message Bus Socket[0m.
[[0;32m OK [0m] Listening on [0;1;39mOpen-iSCSI iscsid Socket[0m.
[[0;32m OK [0m] Listening on [0;1;39mUUID daemon activation socket[0m.
[[0;32m OK [0m] Reached target [0;1;39mSocket Units[0m.
[[0;32m OK [0m] Reached target [0;1;39mBasic System[0m.
[[0;32m OK [0m] Started [0;1;39mD-Bus System Message Bus[0m.
Starting [0;1;39mNetwork Manager[0m...
[[0;32m OK [0m] Started [0;1;39mSave initial kernel messages after boot[0m.
Starting [0;1;39mRemove Stale Onli…t4 Metadata Check Snapshots[0m...
Starting [0;1;39mRecord successful boot for GRUB[0m...
[[0;32m OK [0m] Started [0;1;39mirqbalance daemon[0m.
Starting [0;1;39mKernel crash dump capture service[0m...
Starting [0;1;39mLSB: Execute the …-e command to reboot system[0m...
Starting [0;1;39mDispatcher daemon for systemd-networkd[0m...
Starting [0;1;39mAuthorization Manager[0m...
Starting [0;1;39mSystem Logging Service[0m...
Starting [0;1;39mUser Login Management[0m...
Starting [0;1;39mDisk Manager[0m...
Starting [0;1;39mWPA supplicant[0m...
[[0;32m OK [0m] Started [0;1;39mNetwork Configuration[0m.
Starting [0;1;39mWait for Network to be Configured[0m...
Starting [0;1;39mNetwork Name Resolution[0m...
[[0;32m OK [0m] Finished [0;1;39mRemove Stale Onli…ext4 Metadata Check Snapshots[0m.
[[0;32m OK [0m] Found device [0;1;39m/dev/ttyS0[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Execute the k…c -e command to reboot system[0m.
[[0;32m OK [0m] Started [0;1;39mSystem Logging Service[0m.
[ 9.444413] kdump-tools[471]: Starting kdump-tools:
[ 9.445374] kdump-tools[498]: * Creating symlink /var/lib/kdump/vmlinuz
[ 9.447480] kdump-tools[498]: * Creating symlink /var/lib/kdump/initrd.img
[[0;32m OK [0m] Finished [0;1;39mRecord successful boot for GRUB[0m.
Starting [0;1;39mGRUB failed boot detection[0m...
[[0;32m OK [0m] Listening on [0;1;39mLoad/Save RF …itch Status /dev/rfkill Watch[0m.
[[0;32m OK [0m] Finished [0;1;39mGRUB failed boot detection[0m.
[[0;32m OK [0m] Found device [0;1;39mVirtio network device[0m.
[[0;32m OK [0m] Started [0;1;39mifup for eth0[0m.
[[0;32m OK [0m] Finished [0;1;39mHelper to synchronize boot up for ifupdown[0m.
Starting [0;1;39mRaise network interfaces[0m...
[[0;32m OK [0m] Started [0;1;39mNetwork Name Resolution[0m.
[[0;32m OK [0m] Reached target [0;1;39mHost and Network Name Lookups[0m.
[[0;32m OK [0m] Started [0;1;39mUser Login Management[0m.
[[0;32m OK [0m] Started [0;1;39mWPA supplicant[0m.
Starting [0;1;39mHostname Service[0m...
[[0;32m OK [0m] Started [0;1;39mAuthorization Manager[0m.
Starting [0;1;39mModem Manager[0m...
[[0;32m OK [0m] Started [0;1;39mHostname Service[0m.
[[0;32m OK [0m] Started [0;1;39mNetwork Manager[0m.
Starting [0;1;39mNetwork Manager Wait Online[0m...
Starting [0;1;39mNetwork Manager Script Dispatcher Service[0m...
[[0;32m OK [0m] Started [0;1;39mNetwork Manager Script Dispatcher Service[0m.
[[0;32m OK [0m] Started [0;1;39mModem Manager[0m.
[[0;32m OK [0m] Finished [0;1;39mNetwork Manager Wait Online[0m.
[[0;32m OK [0m] Started [0;1;39mDispatcher daemon for systemd-networkd[0m.
[[0;32m OK [0m] Started [0;1;39mDisk Manager[0m.
[ 11.716064] IPv6: eth0: IPv6 duplicate address fd33:3981:3213:f020:0:5254:82:c0a8 used by 52:54:00:b7:35:71 detected!
[ 11.723339] kdump-tools[498]: * loaded kdump kernel
[[0;32m OK [0m] Finished [0;1;39mKernel crash dump capture service[0m.
[[0m[0;31m* [0m] (1 of 2) A start job is running for… network interfaces (6s / 5min 2s)
M
[K[[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for… network interfaces (7s / 5min 2s)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] (1 of 2) A start job is running for… network interfaces (7s / 5min 2s)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…k to be Configured (8s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…k to be Configured (9s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…k to be Configured (9s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for… network interfaces (9s / 5min 2s)
M
[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mRaise network interfaces[0m.
[KSee 'systemctl status networking.service' for details.
[[0;32m OK [0m] Reached target [0;1;39mNetwork[0m.
Starting [0;1;39mchrony, an NTP client/server[0m...
[[0;32m OK [0m] Started [0;1;39mUnattended Upgrades Shutdown[0m.
[[0;32m OK [0m] Started [0;1;39mchrony, an NTP client/server[0m.
[[0;32m OK [0m] Reached target [0;1;39mSystem Time Synchronized[0m.
[[0;32m OK [0m] Started [0;1;39mDaily apt download activities[0m.
[[0;32m OK [0m] Started [0;1;39mDaily apt upgrade and clean activities[0m.
[[0;32m OK [0m] Started [0;1;39mDaily dpkg database backup timer[0m.
[[0;32m OK [0m] Started [0;1;39mPeriodic ext4 Onli…ata Check for All Filesystems[0m.
[[0;32m OK [0m] Started [0;1;39mDiscard unused blocks once a week[0m.
[[0;32m OK [0m] Started [0;1;39mRefresh fwupd metadata regularly[0m.
[[0;32m OK [0m] Started [0;1;39mDaily rotation of log files[0m.
[[0;32m OK [0m] Started [0;1;39mDaily man-db regeneration[0m.
[[0;32m OK [0m] Started [0;1;39mMessage of the Day[0m.
Starting [0;1;39mMUNGE authentication service[0m...
[[0;32m OK [0m] Started [0;1;39mMUNGE authentication service[0m.
[ [0;31m*[0m] A start job is running for Wait for… to be Configured (12s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (13s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (13s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (14s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (15s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (15s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (16s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (16s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (17s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (18s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (18s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (19s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (19s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (20s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (20s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (22s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (22s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (24s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (25s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (26s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (26s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (27s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (27s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (29s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (29s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (30s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (30s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (31s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (33s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (33s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (34s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (34s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (36s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (36s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (37s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (37s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (40s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (40s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (41s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (41s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (43s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (43s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (44s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (44s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (47s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (47s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (48s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (48s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (50s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (50s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (51s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (51s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (54s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (54s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for… to be Configured (55s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (55s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (57s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (57s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (58s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (58s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M
[K[ [0;31m*[0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M
[K[ [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (2min / no limit)
M
[K[[0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…to be Configured (2min / no limit)
M
[K[[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (2min 1s / no limit)
M
[K[[0m[0;31m* [0m] A start job is running for Wait for…be Configured (2min 1s / no limit)
M
[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mWait for Network to be Configured[0m.
[KSee 'systemctl status systemd-networkd-wait-online.service' for details.
[[0;32m OK [0m] Reached target [0;1;39mNetwork is Online[0m.
[[0;32m OK [0m] Started [0;1;39mDownload data for …ailed at package install time[0m.
[[0;32m OK [0m] Started [0;1;39mCheck to see wheth…w version of Ubuntu available[0m.
[[0;32m OK [0m] Reached target [0;1;39mTimer Units[0m.
Starting [0;1;39mNFS Mount Daemon[0m...
Starting [0;1;39mSamba NMB Daemon[0m...
[[0;32m OK [0m] Reached target [0;1;39mPreparation for Remote File Systems[0m.
Mounting [0;1;39m/autotest[0m...
Mounting [0;1;39m/scratch[0m...
[[0;32m OK [0m] Finished [0;1;39mAvailability of block devices[0m.
Starting [0;1;39mPostfix Mail Transport Agent (instance -)[0m...
Starting [0;1;39mNFS status monitor for NFSv2/3 locking.[0m...
Starting [0;1;39mOpenBSD Secure Shell server[0m...
[[0;32m OK [0m] Started [0;1;39mNFS Mount Daemon[0m.
[[0;32m OK [0m] Started [0;1;39mNFS status monitor for NFSv2/3 locking.[0m.
[[0;32m OK [0m] Started [0;1;39mOpenBSD Secure Shell server[0m.
Starting [0;1;39mNFS server and services[0m...
[ 129.312506] FS-Cache: Loaded
[ 129.365539] FS-Cache: Netfs 'nfs' registered for caching
[ 129.489118] NFSD: Using nfsdcld client tracking operations.
[ 129.490138] NFSD: no clients to reclaim, skipping NFSv4 grace period (net f0000000)
[ 129.511422] NFS: Registering the id_resolver key type
[ 129.512406] Key type id_resolver registered
[ 129.513173] Key type id_legacy registered
[ 129.518202] FS-Cache: Duplicate cookie detected
[ 129.519057] FS-Cache: O-cookie c=00000003 [p=00000002 fl=222 nc=0 na=1]
[ 129.520194] FS-Cache: O-cookie d=000000004bc53828{NFS.server} n=000000001522d9c5
[ 129.521445] FS-Cache: O-key=[16] '0400000002000000020008010af020cc'
[ 129.522528] FS-Cache: N-cookie c=00000004 [p=00000002 fl=2 nc=0 na=1]
[ 129.523615] FS-Cache: N-cookie d=000000004bc53828{NFS.server} n=000000005efbae40
[ 129.524857] FS-Cache: N-key=[16] '0400000002000000020008010af020cc'
[[0;32m OK [0m] Mounted [0;1;39m/autotest[0m.
[[0;32m OK [0m] Mounted [0;1;39m/scratch[0m.
[[0;32m OK [0m] Reached target [0;1;39mRemote File Systems[0m.
Starting [0;1;39mLSB: automatic crash report generation[0m...
Starting [0;1;39mAutomounts filesystems on demand[0m...
[[0;32m OK [0m] Started [0;1;39mRegular background program processing daemon[0m.
Starting [0;1;39mLSB: Load kernel image with kexec[0m...
Starting [0;1;39mLSB: Start/Stop t…mote power manager) daemon.[0m...
Starting [0;1;39mPermit User Sessions[0m...
Starting [0;1;39mLSB: Start vmtouch daemon[0m...
Starting [0;1;39mLSB: Starts or stops the xinetd daemon.[0m...
[[0;32m OK [0m] Started [0;1;39mLSB: Load kernel image with kexec[0m.
[[0;32m OK [0m] Finished [0;1;39mPermit User Sessions[0m.
Starting [0;1;39mHold until boot process finishes up[0m...
Starting [0;1;39mTerminate Plymouth Boot Screen[0m...
[[0;32m OK [0m] Started [0;1;39mLSB: automatic crash report generation[0m.
[[0;32m OK [0m] Finished [0;1;39mHold until boot process finishes up[0m.
[[0;32m OK [0m] Started [0;1;39mSerial Getty on ttyS0[0m.
Starting [0;1;39mSet console scheme[0m...
[[0;32m OK [0m] Finished [0;1;39mTerminate Plymouth Boot Screen[0m.
[[0;32m OK [0m] Finished [0;1;39mNFS server and services[0m.
Starting [0;1;39mNotify NFS peers of a restart[0m...
[[0;32m OK [0m] Started [0;1;39mNotify NFS peers of a restart[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Start/Stop th…remote power manager) daemon.[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Start vmtouch daemon[0m.
[[0;32m OK [0m] Finished [0;1;39mSet console scheme[0m.
[[0;32m OK [0m] Created slice [0;1;39mSlice /system/getty[0m.
[[0;32m OK [0m] Started [0;1;39mGetty on tty1[0m.
[[0;32m OK [0m] Reached target [0;1;39mLogin Prompts[0m.
[[0;32m OK [0m] Started [0;1;39mLSB: Starts or stops the xinetd daemon.[0m.
[[0;32m OK [0m] Started [0;1;39mAutomounts filesystems on demand[0m.
[[0;32m OK [0m] Started [0;1;39mSamba NMB Daemon[0m.
Starting [0;1;39mSamba SMB Daemon[0m...
[[0;32m OK [0m] Started [0;1;39mSamba SMB Daemon[0m.
[[0;32m OK [0m] Started [0;1;39mPostfix Mail Transport Agent (instance -)[0m.
Starting [0;1;39mPostfix Mail Transport Agent[0m...
[[0;32m OK [0m] Finished [0;1;39mPostfix Mail Transport Agent[0m.
[[0;32m OK [0m] Reached target [0;1;39mMulti-User System[0m.
[[0;32m OK [0m] Reached target [0;1;39mGraphical Interface[0m.
Starting [0;1;39mRecord Runlevel Change in UTMP[0m...
[[0;32m OK [0m] Finished [0;1;39mRecord Runlevel Change in UTMP[0m.
Ubuntu 22.04 LTS trevis-106vm14.trevis.whamcloud.com ttyS0
trevis-106vm14 login: [ 146.384994] Autotest: reachable and ready (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 242.066482] Autotest: Cluster successfully rebooted, running reset steps (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 259.486784] Autotest: Cluster reset complete! (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 260.688351] Autotest: Starting test framework on controller trevis-106vm14 (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 262.709512] Autotest: Test framework started (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 262.846766] libcfs: loading out-of-tree module taints kernel.
[ 262.848276] libcfs: module verification failed: signature and/or required key missing - tainting kernel
[ 262.853953] Key type ._llcrypt registered
[ 262.854702] Key type .llcrypt registered
[ 263.062624] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[ 263.756210] Autotest: Test framework pid=2859 (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 264.868135] Autotest: Test running for 0 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 267.643693] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 267.655205] Lustre: DEBUG MARKER: /usr/sbin/lctl lustre_build_version 2>/dev/null
[ 270.822042] Lustre: DEBUG MARKER: cat /etc/system-release
[ 270.832950] Lustre: DEBUG MARKER: test -r /etc/os-release
[ 270.843117] Lustre: DEBUG MARKER: cat /etc/os-release
[ 271.083304] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 271.083304] if [ $running -ne 0 ] ; then
[ 271.083304] echo Stopping client $(hostname) /mnt/lustre opts:-f;
[ 271.083304] lsof /mnt/lustre || need_kill=no;
[ 271.083304] if [ x-f != x -a x$need_kill != xno ]; then
[ 271.083304] pids=$(lsof -t /mnt/lustre | sort -u);
[ 271.083304]
[ 274.552023] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 274.552023] if [ $running -ne 0 ] ; then
[ 274.552023] echo Stopping client $(hostname) /mnt/lustre2 opts:-f;
[ 274.552023] lsof /mnt/lustre2 || need_kill=no;
[ 274.552023] if [ x-f != x -a x$need_kill != xno ]; then
[ 274.552023] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 289.580681] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 289.759367] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 289.760652] LNet: Accept all, port 7988
[ 291.305220] Key type lgssc registered
[ 291.501640] Lustre: Echo OBD driver; http://www.lustre.org/
[ 436.273354] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 436.273354] mpts=$(mount | grep -c /mnt/lustre' ');
[ 436.273354] if [ $running -ne $mpts ]; then
[ 436.273354] echo $(hostname) env are INSANE!;
[ 436.273354] exit 1;
[ 436.273354] fi
[ 436.910090] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 436.910090] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 436.910090] if [ $running -ne $mpts ]; then
[ 436.910090] echo $(hostname) env are INSANE!;
[ 436.910090] exit 1;
[ 436.910090] fi
[ 542.966759] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 542.992376] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 544.072463] Lustre: Mounted lustre-client
[ 544.282935] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 544.899528] Lustre: DEBUG MARKER:
[ 544.899528] running=$(mount | grep -c /mnt/lustre' ');
[ 544.899528] rc=0;
[ 544.899528] if [ $running -eq 0 ] ; then
[ 544.899528] mkdir -p /mnt/lustre;
[ 544.899528] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 544.899528] rc=$?;
[ 544.899528] fi;
[ 544.899528] exit $rc
[ 565.826512] Autotest: Test running for 5 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 612.120045] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 629.097066] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 629.428159] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 629.789921] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 630.602117] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 630.628942] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 630.642739] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 631.001429] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 632.014982] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 633.028735] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 633.413826] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[ 633.424027] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[ 641.064801] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 641.335755] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Client: 2.16.50.134
[ 641.661186] Lustre: DEBUG MARKER: Client: 2.16.50.134
[ 642.394824] Lustre: DEBUG MARKER: /usr/sbin/lctl mark MDS: 2.16.50.134
[ 642.729322] Lustre: DEBUG MARKER: MDS: 2.16.50.134
[ 643.474339] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OSS: 2.16.50.134
[ 643.799734] Lustre: DEBUG MARKER: OSS: 2.16.50.134
[ 644.186164] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: sanity-sec ============----- Mon Dec 16 21:26:28 UTC 2024
[ 644.518674] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-sec ============----- Mon Dec 16 21:26:28 UTC 2024
[ 646.207813] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 649.400263] Lustre: DEBUG MARKER: cat /etc/system-release
[ 649.411372] Lustre: DEBUG MARKER: test -r /etc/os-release
[ 649.421704] Lustre: DEBUG MARKER: cat /etc/os-release
[ 650.489483] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests:
[ 650.822128] Lustre: DEBUG MARKER: excepting tests:
[ 651.184928] Lustre: DEBUG MARKER: /usr/sbin/lctl mark skipping tests SLOW=no: 26
[ 651.515092] Lustre: DEBUG MARKER: skipping tests SLOW=no: 26
[ 651.618076] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 24s idle
[ 652.023911] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: start setup 21:26:36 \(1734384396\) ===
[ 652.354334] Lustre: DEBUG MARKER: === sanity-sec: start setup 21:26:36 (1734384396) ===
[ 652.693224] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 653.776996] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing check_config_client \/mnt\/lustre
[ 654.128229] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing check_config_client /mnt/lustre
[ 658.390249] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 658.390249] mpts=$(mount | grep -c /mnt/lustre' ');
[ 658.390249] if [ $running -ne $mpts ]; then
[ 658.390249] echo $(hostname) env are INSANE!;
[ 658.390249] exit 1;
[ 658.390249] fi
[ 659.014136] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 659.014136] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 659.014136] if [ $running -ne $mpts ]; then
[ 659.014136] echo $(hostname) env are INSANE!;
[ 659.014136] exit 1;
[ 659.014136] fi
[ 667.145403] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 667.475908] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 667.838182] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 668.648258] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 674.101357] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: finish setup 21:26:58 \(1734384418\) ===
[ 674.422955] Lustre: DEBUG MARKER: === sanity-sec: finish setup 21:26:58 (1734384418) ===
[ 676.221980] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 0: uid permission ======================================================================================= 21:27:00 \(1734384420\)
[ 676.550525] Lustre: DEBUG MARKER: == sanity-sec test 0: uid permission ======================================================================================= 21:27:00 (1734384420)
[ 678.379749] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 1: setuid\/gid ======================================================================================= 21:27:02 \(1734384422\)
[ 678.707576] Lustre: DEBUG MARKER: == sanity-sec test 1: setuid/gid ======================================================================================= 21:27:02 (1734384422)
[ 679.078403] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_1 without GSS support.
[ 679.400963] Lustre: DEBUG MARKER: SKIP: sanity-sec test_1 without GSS support.
[ 679.794847] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 4: set supplementary group ========================================================================= 21:27:03 \(1734384423\)
[ 680.123765] Lustre: DEBUG MARKER: == sanity-sec test 4: set supplementary group ========================================================================= 21:27:03 (1734384423)
[ 690.810676] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 7: nodemap create and delete ========== 21:27:15 \(1734384435\)
[ 691.142051] Lustre: DEBUG MARKER: == sanity-sec test 7: nodemap create and delete ========== 21:27:15 (1734384435)
[ 697.698193] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 21s idle
[ 697.703546] Lustre: Skipped 6 previous similar messages
[ 715.168841] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 8: nodemap reject duplicates ========== 21:27:39 \(1734384459\)
[ 715.497033] Lustre: DEBUG MARKER: == sanity-sec test 8: nodemap reject duplicates ========== 21:27:39 (1734384459)
[ 745.190037] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 9: nodemap range add ================== 21:28:09 \(1734384489\)
[ 745.517080] Lustre: DEBUG MARKER: == sanity-sec test 9: nodemap range add ================== 21:28:09 (1734384489)
[ 773.572363] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10a: nodemap reject duplicate ranges == 21:28:37 \(1734384517\)
[ 773.906081] Lustre: DEBUG MARKER: == sanity-sec test 10a: nodemap reject duplicate ranges == 21:28:37 (1734384517)
[ 804.027945] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10b: delete range from the correct nodemap ========================================================== 21:29:08 \(1734384548\)
[ 804.360789] Lustre: DEBUG MARKER: == sanity-sec test 10b: delete range from the correct nodemap ========================================================== 21:29:08 (1734384548)
[ 809.434934] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10c: verfify contiguous range support ========================================================== 21:29:13 \(1734384553\)
[ 809.773474] Lustre: DEBUG MARKER: == sanity-sec test 10c: verfify contiguous range support ========================================================== 21:29:13 (1734384553)
[ 813.489314] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10d: verfify nodemap range format \'\*@\<net\>\' support ========================================================== 21:29:17 \(1734384557\)
[ 813.826235] Lustre: DEBUG MARKER: == sanity-sec test 10d: verfify nodemap range format '*@<net>' support ========================================================== 21:29:17 (1734384557)
[ 817.486619] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 11: nodemap modify ==================== 21:29:21 \(1734384561\)
[ 817.821817] Lustre: DEBUG MARKER: == sanity-sec test 11: nodemap modify ==================== 21:29:21 (1734384561)
[ 845.974302] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 12: nodemap set squash ids ============ 21:29:50 \(1734384590\)
[ 846.308212] Lustre: DEBUG MARKER: == sanity-sec test 12: nodemap set squash ids ============ 21:29:50 (1734384590)
[ 866.041133] Autotest: Test running for 10 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 873.400699] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 13: test nids ========================= 21:30:17 \(1734384617\)
[ 873.732766] Lustre: DEBUG MARKER: == sanity-sec test 13: test nids ========================= 21:30:17 (1734384617)
[ 904.014681] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 14: test default nodemap nid lookup === 21:30:48 \(1734384648\)
[ 904.367511] Lustre: DEBUG MARKER: == sanity-sec test 14: test default nodemap nid lookup === 21:30:48 (1734384648)
[ 932.635306] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 15: test id mapping =================== 21:31:16 \(1734384676\)
[ 932.968112] Lustre: DEBUG MARKER: == sanity-sec test 15: test id mapping =================== 21:31:16 (1734384676)
[ 1007.443368] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 16: test nodemap all_off fileops ====== 21:32:31 \(1734384751\)
[ 1007.782767] Lustre: DEBUG MARKER: == sanity-sec test 16: test nodemap all_off fileops ====== 21:32:31 (1734384751)
[ 1008.309272] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1021.924457] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d16.sanity-sec
[ 1021.936058] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d16.sanity-sec
[ 1021.958958] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d16.sanity-sec
[ 1022.646147] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1026.183367] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1026.226038] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1026.841949] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1027.094390] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1027.716110] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1027.739105] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[ 1029.072487] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1029.091865] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1038.501751] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1039.206360] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1039.221929] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1039.839743] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1040.456911] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1040.480023] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[ 1041.797185] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1041.817990] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1049.820369] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1052.632976] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d16.sanity-sec
[ 1052.644099] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d16.sanity-sec
[ 1052.654684] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d16.sanity-sec
[ 1053.348244] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1055.458555] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1055.474458] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1056.087955] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1056.103122] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1056.721180] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1056.740459] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[ 1058.103144] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1058.122328] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1067.473415] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1068.179687] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1068.194815] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1068.806679] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1069.412939] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1069.434021] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[ 1070.868388] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1070.888894] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[ 1080.248514] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1087.256296] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 17: test nodemap trusted_noadmin fileops ========================================================== 21:33:51 \(1734384831\)
[ 1087.593363] Lustre: DEBUG MARKER: == sanity-sec test 17: test nodemap trusted_noadmin fileops ========================================================== 21:33:51 (1734384831)
[ 1088.122791] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1102.179186] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 22s idle
[ 1102.180585] Lustre: Skipped 3 previous similar messages
[ 1113.398670] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[ 1113.409781] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[ 1113.420511] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d17.sanity-sec
[ 1114.125428] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1123.945447] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1123.980430] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1124.594190] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1125.210297] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1125.233304] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1126.431596] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1126.450922] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1137.218888] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1137.916569] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1137.939198] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1138.563599] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1139.179818] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1139.200892] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1140.563297] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1140.583886] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1148.572573] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1154.377212] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[ 1154.403996] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[ 1154.415015] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d17.sanity-sec
[ 1155.105469] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1160.193676] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1160.209998] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1160.816985] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1161.431760] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1161.452624] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1162.964634] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1162.984569] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1167.231427] Autotest: Test running for 15 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1172.371578] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1172.995530] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1173.010974] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1173.631860] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1174.247637] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1174.268352] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1175.787203] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1175.814259] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1183.802621] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1192.123900] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[ 1192.138325] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[ 1192.149571] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d17.sanity-sec
[ 1192.850706] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1197.953302] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1197.969376] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1198.585924] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1199.199923] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1199.219709] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1201.071201] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1201.090971] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1209.103201] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1209.724748] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1209.739760] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1210.351492] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1210.972529] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1210.994374] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1212.671773] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1212.692458] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1220.760343] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1226.650026] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[ 1226.663985] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[ 1226.681254] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d17.sanity-sec
[ 1227.388759] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1232.485947] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1232.502076] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1233.107987] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1233.718506] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1233.974272] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1235.529818] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1235.549770] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1243.529939] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1244.146116] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1244.160965] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1244.768592] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1245.373112] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1245.394091] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[ 1246.914684] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1246.935724] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[ 1254.913399] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1266.578392] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 18: test nodemap mapped_noadmin fileops ========================================================== 21:36:50 \(1734385010\)
[ 1266.908925] Lustre: DEBUG MARKER: == sanity-sec test 18: test nodemap mapped_noadmin fileops ========================================================== 21:36:50 (1734385010)
[ 1267.438334] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1276.259637] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 21s idle
[ 1276.261071] Lustre: Skipped 6 previous similar messages
[ 1294.424667] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d18.sanity-sec
[ 1294.436357] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d18.sanity-sec
[ 1294.452503] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d18.sanity-sec
[ 1295.141807] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1303.160095] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1303.233738] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1303.847298] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1304.457235] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1304.480353] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[ 1305.786536] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1305.806669] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1316.616156] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1317.333016] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1317.348107] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1317.974794] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1318.598688] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1318.621670] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[ 1320.201165] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1320.222204] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1332.348808] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1338.245051] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d18.sanity-sec
[ 1338.258899] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d18.sanity-sec
[ 1338.269790] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d18.sanity-sec
[ 1338.955636] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1344.025210] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1344.041435] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1344.649212] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1345.271002] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1345.290157] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[ 1346.445210] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1346.464410] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1354.465164] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1355.088587] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1355.704294] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1355.719495] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1356.339750] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1356.361326] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[ 1357.993640] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1358.013875] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[ 1367.421580] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1377.394944] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 19: test nodemap trusted_admin fileops ========================================================== 21:38:41 \(1734385121\)
[ 1377.729473] Lustre: DEBUG MARKER: == sanity-sec test 19: test nodemap trusted_admin fileops ========================================================== 21:38:41 (1734385121)
[ 1378.256171] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1388.899862] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 22s idle
[ 1388.901325] Lustre: Skipped 6 previous similar messages
[ 1403.480175] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d19.sanity-sec
[ 1403.491582] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d19.sanity-sec
[ 1403.502794] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d19.sanity-sec
[ 1404.197561] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1412.214941] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1412.258715] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1412.876385] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1412.976697] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1413.597225] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1413.618679] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[ 1414.963250] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1414.983540] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1424.438996] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1425.141813] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1425.157702] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1425.768173] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1426.376763] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1426.397686] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[ 1427.856671] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1427.877691] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1435.916262] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1441.777359] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d19.sanity-sec
[ 1441.788945] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d19.sanity-sec
[ 1441.800408] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d19.sanity-sec
[ 1442.487185] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1447.569986] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1447.585647] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1448.208360] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1448.223924] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1448.840838] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1448.860994] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[ 1450.137741] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1450.157081] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1459.566459] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1460.277010] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1460.294332] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1460.908245] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1461.524543] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1461.547152] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[ 1462.855341] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1462.878893] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[ 1468.241294] Autotest: Test running for 20 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1469.585520] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1479.449713] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 20: test nodemap mapped_admin fileops ========================================================== 21:40:23 \(1734385223\)
[ 1479.787463] Lustre: DEBUG MARKER: == sanity-sec test 20: test nodemap mapped_admin fileops ========================================================== 21:40:23 (1734385223)
[ 1480.312750] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1491.300143] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 21s idle
[ 1491.301555] Lustre: Skipped 6 previous similar messages
[ 1507.375439] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d20.sanity-sec
[ 1507.392813] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d20.sanity-sec
[ 1507.403898] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d20.sanity-sec
[ 1508.096535] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1516.057699] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1516.190188] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1516.810729] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1516.942082] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1517.562010] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1517.584215] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[ 1518.840842] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1518.860559] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1526.867074] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1527.485251] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1527.501019] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1528.117400] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1528.733761] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1528.755375] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[ 1530.127095] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1530.147341] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1538.156947] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1544.078754] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d20.sanity-sec
[ 1544.090095] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d20.sanity-sec
[ 1544.100659] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d20.sanity-sec
[ 1544.787694] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1549.888270] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1549.903144] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1550.524406] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1550.539378] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1551.152655] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1551.171965] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[ 1552.247539] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1552.267745] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1561.716253] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1562.421499] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1563.044184] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1563.059164] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1563.682431] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1563.704697] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[ 1565.122374] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1565.141953] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[ 1577.305054] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1587.364460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 21: test nodemap mapped_trusted_noadmin fileops ========================================================== 21:42:11 \(1734385331\)
[ 1587.692824] Lustre: DEBUG MARKER: == sanity-sec test 21: test nodemap mapped_trusted_noadmin fileops ========================================================== 21:42:11 (1734385331)
[ 1588.217490] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1598.820401] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 22s idle
[ 1598.821790] Lustre: Skipped 6 previous similar messages
[ 1613.141361] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d21.sanity-sec
[ 1613.153022] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d21.sanity-sec
[ 1613.163811] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d21.sanity-sec
[ 1613.855705] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1621.826926] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1621.879210] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1622.495610] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1623.116822] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1623.138927] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[ 1624.378316] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1624.397659] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1633.817654] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1634.537495] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1634.552745] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1635.165456] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1635.775495] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1635.797558] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[ 1637.757231] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1637.778301] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1648.531936] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1654.434937] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d21.sanity-sec
[ 1654.449235] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d21.sanity-sec
[ 1654.460187] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d21.sanity-sec
[ 1655.156103] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1660.262959] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1660.278822] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1660.897843] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1661.516496] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1661.665739] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[ 1662.962247] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1662.981843] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1671.018403] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1671.650006] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1671.666131] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1672.286984] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1672.913931] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1672.935584] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[ 1674.368023] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1674.389231] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[ 1682.390248] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1692.371140] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 22: test nodemap mapped_trusted_admin fileops ========================================================== 21:43:56 \(1734385436\)
[ 1692.707269] Lustre: DEBUG MARKER: == sanity-sec test 22: test nodemap mapped_trusted_admin fileops ========================================================== 21:43:56 (1734385436)
[ 1693.239914] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1706.340670] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 24s idle
[ 1706.342078] Lustre: Skipped 6 previous similar messages
[ 1716.390493] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d22.sanity-sec
[ 1716.401478] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d22.sanity-sec
[ 1716.411985] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d22.sanity-sec
[ 1717.103260] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1722.584902] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1722.804044] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1723.417493] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1723.655294] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1724.268894] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1724.291054] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[ 1725.793678] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1725.813291] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1736.580113] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1737.283773] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1737.303987] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1737.917969] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1738.539233] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1738.560948] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[ 1739.941050] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1739.964318] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1747.982586] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1753.837790] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d22.sanity-sec
[ 1753.849582] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d22.sanity-sec
[ 1753.860688] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d22.sanity-sec
[ 1754.555252] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 1759.727255] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1759.742881] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1760.356454] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1760.370929] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1760.983021] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1761.005449] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[ 1762.430510] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1762.449853] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1769.105700] Autotest: Test running for 25 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1771.891914] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[ 1772.512861] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1772.527800] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1773.138101] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1773.761668] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1773.782037] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[ 1775.328103] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1775.348556] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[ 1787.513358] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[ 1799.410727] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 23a: test mapped regular ACLs ========= 21:45:43 \(1734385543\)
[ 1799.744190] Lustre: DEBUG MARKER: == sanity-sec test 23a: test mapped regular ACLs ========= 21:45:43 (1734385543)
[ 1800.272424] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 1808.740877] Lustre: lustre-OST0000-osc-ffff9509057e9800: disconnect after 22s idle
[ 1808.742310] Lustre: Skipped 6 previous similar messages
[ 1825.169989] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[ 1825.181202] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[ 1825.191923] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[ 1833.076568] Lustre: DEBUG MARKER: runas -u 60000 touch /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1833.297900] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1834.011944] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1834.422945] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1834.563191] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1834.902972] bash (90064): drop_caches: 3
[ 1839.593990] Lustre: DEBUG MARKER: runas -u 60000 setfacl -m 60000:rwx /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1839.625345] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1840.354329] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1841.791868] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1841.927239] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1842.029872] bash (90342): drop_caches: 3
[ 1853.399948] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[ 1853.418080] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[ 1853.435268] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[ 1861.323629] Lustre: DEBUG MARKER: runas -u 60002 touch /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1861.386229] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1862.105816] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1862.787003] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1862.926126] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1863.012203] bash (91246): drop_caches: 3
[ 1867.276204] Lustre: DEBUG MARKER: runas -u 60002 setfacl -m 60002:rwx /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1867.294125] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1868.003419] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1868.329101] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1868.466249] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1868.544831] bash (91523): drop_caches: 3
[ 1878.200270] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[ 1878.218432] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[ 1878.229562] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[ 1890.704463] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1890.730978] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1891.444575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1891.780176] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1891.918433] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1892.042420] bash (92486): drop_caches: 3
[ 1895.766829] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1895.782093] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1896.497174] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1896.832530] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1896.974373] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1897.107276] bash (92763): drop_caches: 3
[ 1902.906335] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[ 1902.923722] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[ 1902.935016] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[ 1915.084675] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1915.100834] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1915.815341] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1916.146712] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1916.286860] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1916.369884] bash (93667): drop_caches: 3
[ 1928.665752] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[ 1928.683663] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[ 1928.694831] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[ 1940.376578] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1940.393177] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1941.137767] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1941.463035] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1941.603373] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1941.705966] bash (94831): drop_caches: 3
[ 1946.067088] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1946.085497] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1946.803721] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1947.136313] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1947.270802] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1947.495968] bash (95108): drop_caches: 3
[ 1953.347976] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[ 1953.479076] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[ 1953.490884] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[ 1965.462828] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1965.479292] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1966.200328] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1966.535648] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1966.680311] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1966.791318] bash (96012): drop_caches: 3
[ 1967.461251] Lustre: lustre-OST0001-osc-ffff9509057e9800: disconnect after 20s idle
[ 1967.462628] Lustre: Skipped 10 previous similar messages
[ 1972.413785] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[ 1972.446511] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 1973.159973] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1973.488332] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing cancel_lru_locks
[ 1973.629975] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[ 1973.768882] bash (96289): drop_caches: 3
[ 1983.639386] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 23b: test mapped default ACLs ========= 21:48:47 \(1734385727\)
[ 1983.970113] Lustre: DEBUG MARKER: == sanity-sec test 23b: test mapped default ACLs ========= 21:48:47 (1734385727)
[ 1984.496259] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 2011.909537] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23b.sanity-sec
[ 2011.920993] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23b.sanity-sec
[ 2011.932132] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23b.sanity-sec
[ 2011.944125] Lustre: DEBUG MARKER: setfacl -R -d -m group:sanityusr:rwx /mnt/lustre/d23b.sanity-sec
[ 2011.958041] Lustre: DEBUG MARKER: sync && stat /mnt/lustre/d23b.sanity-sec > /dev/null
[ 2012.016571] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -t4 -n ldlm.namespaces.*.lru_size=clear
[ 2012.028308] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23b.sanity-sec
[ 2012.041170] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23b.sanity-sec
[ 2026.418341] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 24: check nodemap proc files for LBUGs and Oopses ========================================================== 21:49:30 \(1734385770\)
[ 2026.755392] Lustre: DEBUG MARKER: == sanity-sec test 24: check nodemap proc files for LBUGs and Oopses ========================================================== 21:49:30 (1734385770)
[ 2027.263029] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 2053.095849] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 25: test save and reload nodemap config ========================================================== 21:49:57 \(1734385797\)
[ 2053.437273] Lustre: DEBUG MARKER: == sanity-sec test 25: test save and reload nodemap config ========================================================== 21:49:57 (1734385797)
[ 2053.767367] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2053.767367] if [ $running -ne 0 ] ; then
[ 2053.767367] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2053.767367] lsof /mnt/lustre || need_kill=no;
[ 2053.767367] if [ x != x -a x$need_kill != xno ]; then
[ 2053.767367] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2053.767367] if
[ 2054.592553] Lustre: Unmounted lustre-client
[ 2055.438826] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 2069.970167] Autotest: Test running for 30 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2076.642042] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: start setup 21:50:20 \(1734385820\) ===
[ 2076.974001] Lustre: DEBUG MARKER: === sanity-sec: start setup 21:50:20 (1734385820) ===
[ 2080.845794] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2080.845794] mpts=$(mount | grep -c /mnt/lustre' ');
[ 2080.845794] if [ $running -ne $mpts ]; then
[ 2080.845794] echo $(hostname) env are INSANE!;
[ 2080.845794] exit 1;
[ 2080.845794] fi
[ 2081.513874] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 2081.513874] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 2081.513874] if [ $running -ne $mpts ]; then
[ 2081.513874] echo $(hostname) env are INSANE!;
[ 2081.513874] exit 1;
[ 2081.513874] fi
[ 2108.720102] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2108.733316] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2108.781524] Lustre: Mounted lustre-client
[ 2108.979235] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2109.740933] Lustre: DEBUG MARKER:
[ 2109.740933] running=$(mount | grep -c /mnt/lustre' ');
[ 2109.740933] rc=0;
[ 2109.740933] if [ $running -eq 0 ] ; then
[ 2109.740933] mkdir -p /mnt/lustre;
[ 2109.740933] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 2109.740933] rc=$?;
[ 2109.740933] fi;
[ 2109.740933] exit $rc
[ 2110.424609] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2116.280650] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 2116.615823] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 2116.990245] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 2117.795826] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 2118.188461] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[ 2118.199074] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[ 2127.237097] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: finish setup 21:51:11 \(1734385871\) ===
[ 2127.567062] Lustre: DEBUG MARKER: === sanity-sec: finish setup 21:51:11 (1734385871) ===
[ 2127.871078] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2127.871078] if [ $running -ne 0 ] ; then
[ 2127.871078] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2127.871078] lsof /mnt/lustre || need_kill=no;
[ 2127.871078] if [ x != x -a x$need_kill != xno ]; then
[ 2127.871078] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2127.871078] if
[ 2128.670135] Lustre: Unmounted lustre-client
[ 2138.584418] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2139.219177] Lustre: DEBUG MARKER:
[ 2139.219177] running=$(mount | grep -c /mnt/lustre' ');
[ 2139.219177] rc=0;
[ 2139.219177] if [ $running -eq 0 ] ; then
[ 2139.219177] mkdir -p /mnt/lustre;
[ 2139.219177] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 2139.219177] rc=$?;
[ 2139.219177] fi;
[ 2139.219177] exit $rc
[ 2139.595041] Lustre: Mounted lustre-client
[ 2139.898973] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2143.784674] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_26 skipping SLOW test 26
[ 2144.112765] Lustre: DEBUG MARKER: SKIP: sanity-sec test_26 skipping SLOW test 26
[ 2144.509664] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 27a: test fileset in various nodemaps ========================================================== 21:51:28 \(1734385888\)
[ 2144.839438] Lustre: DEBUG MARKER: == sanity-sec test 27a: test fileset in various nodemaps ========================================================== 21:51:28 (1734385888)
[ 2156.148067] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[ 2156.161549] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[ 2156.178283] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default/subsubdir_default
[ 2156.189952] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default/subsubdir_default/this_is_subsubdir_default
[ 2156.885867] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 2166.076252] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2166.076252] if [ $running -ne 0 ] ; then
[ 2166.076252] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2166.076252] lsof /mnt/lustre || need_kill=no;
[ 2166.076252] if [ x != x -a x$need_kill != xno ]; then
[ 2166.076252] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2166.076252] if
[ 2166.981758] Lustre: Unmounted lustre-client
[ 2167.533831] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 2167.533831] if [ $running -ne 0 ] ; then
[ 2167.533831] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 2167.533831] lsof /mnt/lustre2 || need_kill=no;
[ 2167.533831] if [ x != x -a x$need_kill != xno ]; then
[ 2167.533831] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 2167.533831]
[ 2198.344774] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[ 2198.344774] lctl dl | grep ' ST ' || true
[ 2198.727484] Key type lgssc unregistered
[ 2198.797890] LNet: 110563:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 2198.821931] LNet: Removed LNI 10.240.44.153@tcp
[ 2199.046247] Key type .llcrypt unregistered
[ 2199.047169] Key type ._llcrypt unregistered
[ 2204.144119] Key type ._llcrypt registered
[ 2204.144973] Key type .llcrypt registered
[ 2204.219351] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[ 2205.051702] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 2205.121394] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 2205.122574] LNet: Accept all, port 7988
[ 2206.661783] Key type lgssc registered
[ 2206.853844] Lustre: Echo OBD driver; http://www.lustre.org/
[ 2277.691138] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2277.701297] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2277.701297] umount /mnt/lustre
[ 2277.716665] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2278.751441] Lustre: Mounted lustre-client
[ 2278.759187] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 2278.770773] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//
[ 2278.781633] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2278.781633] umount /mnt/lustre
[ 2278.824066] Lustre: Unmounted lustre-client
[ 2278.830595] Lustre: DEBUG MARKER:
[ 2278.830595] running=$(mount | grep -c /mnt/lustre' ');
[ 2278.830595] rc=0;
[ 2278.830595] if [ $running -eq 0 ] ; then
[ 2278.830595] mkdir -p /mnt/lustre;
[ 2278.830595] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/ /mnt/lustre;
[ 2278.830595] rc=$?;
[ 2278.830595] fi;
[ 2278.830595] exit $rc
[ 2278.882380] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2278.909971] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[ 2278.922503] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2278.922503] if [ $running -ne 0 ] ; then
[ 2278.922503] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2278.922503] lsof /mnt/lustre || need_kill=no;
[ 2278.922503] if [ x != x -a x$need_kill != xno ]; then
[ 2278.922503] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2278.922503] if
[ 2279.115876] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2279.126309] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2279.126309] umount /mnt/lustre
[ 2279.137460] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2279.180869] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 2279.193095] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//subsubdir_default
[ 2279.206327] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2279.206327] umount /mnt/lustre
[ 2279.245557] Lustre: DEBUG MARKER:
[ 2279.245557] running=$(mount | grep -c /mnt/lustre' ');
[ 2279.245557] rc=0;
[ 2279.245557] if [ $running -eq 0 ] ; then
[ 2279.245557] mkdir -p /mnt/lustre;
[ 2279.245557] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/subsubdir_default /mnt/lustre;
[ 2279.245557] rc=$?;
[ 2279.245557] fi;
[ 2279.245557] exit $rc
[ 2279.288585] Lustre: Mounted lustre-client
[ 2279.289382] Lustre: Skipped 2 previous similar messages
[ 2279.294907] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2279.322728] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_subsubdir_default
[ 2291.439080] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2291.439080] if [ $running -ne 0 ] ; then
[ 2291.439080] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2291.439080] lsof /mnt/lustre || need_kill=no;
[ 2291.439080] if [ x != x -a x$need_kill != xno ]; then
[ 2291.439080] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2291.439080] if
[ 2291.608835] Lustre: Unmounted lustre-client
[ 2291.609679] Lustre: Skipped 2 previous similar messages
[ 2291.616113] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2291.626285] Lustre: DEBUG MARKER:
[ 2291.626285] running=$(mount | grep -c /mnt/lustre' ');
[ 2291.626285] rc=0;
[ 2291.626285] if [ $running -eq 0 ] ; then
[ 2291.626285] mkdir -p /mnt/lustre;
[ 2291.626285] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 2291.626285] rc=$?;
[ 2291.626285] fi;
[ 2291.626285] exit $rc
[ 2291.674810] Lustre: Mounted lustre-client
[ 2291.680497] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2291.709238] Lustre: DEBUG MARKER: test -d /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[ 2297.521367] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[ 2298.235091] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 2318.837604] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2318.849366] Lustre: DEBUG MARKER:
[ 2318.849366] running=$(mount | grep -c /mnt/lustre' ');
[ 2318.849366] rc=0;
[ 2318.849366] if [ $running -eq 0 ] ; then
[ 2318.849366] mkdir -p /mnt/lustre;
[ 2318.849366] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 2318.849366] rc=$?;
[ 2318.849366] fi;
[ 2318.849366] exit $rc
[ 2318.865070] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2329.202818] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 2370.160702] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[ 2370.182175] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[ 2370.278192] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0/subsubdir_c0
[ 2370.290288] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0/subsubdir_c0/this_is_subsubdir_c0
[ 2371.000170] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 2375.934278] Autotest: Test running for 35 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2417.492641] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2417.492641] if [ $running -ne 0 ] ; then
[ 2417.492641] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2417.492641] lsof /mnt/lustre || need_kill=no;
[ 2417.492641] if [ x != x -a x$need_kill != xno ]; then
[ 2417.492641] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2417.492641] if
[ 2418.405516] Lustre: Unmounted lustre-client
[ 2418.933936] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 2418.933936] if [ $running -ne 0 ] ; then
[ 2418.933936] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 2418.933936] lsof /mnt/lustre2 || need_kill=no;
[ 2418.933936] if [ x != x -a x$need_kill != xno ]; then
[ 2418.933936] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 2418.933936]
[ 2448.709987] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[ 2448.709987] lctl dl | grep ' ST ' || true
[ 2449.092560] Key type lgssc unregistered
[ 2449.154470] LNet: 120938:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 2450.198511] LNet: Removed LNI 10.240.44.153@tcp
[ 2450.410567] Key type .llcrypt unregistered
[ 2450.411424] Key type ._llcrypt unregistered
[ 2455.525214] Key type ._llcrypt registered
[ 2455.526108] Key type .llcrypt registered
[ 2455.574799] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[ 2456.405517] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 2456.459632] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 2456.460809] LNet: Accept all, port 7988
[ 2457.994353] Key type lgssc registered
[ 2458.144799] Lustre: Echo OBD driver; http://www.lustre.org/
[ 2541.863752] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2541.874455] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2541.874455] umount /mnt/lustre
[ 2541.885225] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2548.076144] Lustre: Mounted lustre-client
[ 2548.084266] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 2548.096656] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//
[ 2548.107530] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2548.107530] umount /mnt/lustre
[ 2548.147241] Lustre: Unmounted lustre-client
[ 2548.153304] Lustre: DEBUG MARKER:
[ 2548.153304] running=$(mount | grep -c /mnt/lustre' ');
[ 2548.153304] rc=0;
[ 2548.153304] if [ $running -eq 0 ] ; then
[ 2548.153304] mkdir -p /mnt/lustre;
[ 2548.153304] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/ /mnt/lustre;
[ 2548.153304] rc=$?;
[ 2548.153304] fi;
[ 2548.153304] exit $rc
[ 2548.204993] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2548.234413] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[ 2548.247263] Lustre: DEBUG MARKER: /usr/bin/lfs setquota -p 1 -b 10000 -B 11000 -i 0 -I 0 /mnt/lustre
[ 2548.260582] Lustre: DEBUG MARKER: /usr/bin/lfs setquota -p 2 -b 10000 -B 11000 -i 0 -I 0 /mnt/lustre
[ 2548.269507] LustreError: 123442:0:(mdc_request.c:2134:mdc_quotactl()) lustre-MDT0000-mdc-ffff950910353000: ptlrpc_queue_wait failed: rc = -1
[ 2548.275275] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2548.275275] if [ $running -ne 0 ] ; then
[ 2548.275275] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2548.275275] lsof /mnt/lustre || need_kill=no;
[ 2548.275275] if [ x != x -a x$need_kill != xno ]; then
[ 2548.275275] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2548.275275] if
[ 2548.478031] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2548.488209] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2548.488209] umount /mnt/lustre
[ 2548.499527] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2548.545044] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 2548.556717] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//subsubdir_c0
[ 2548.569688] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 2548.569688] umount /mnt/lustre
[ 2548.610472] Lustre: DEBUG MARKER:
[ 2548.610472] running=$(mount | grep -c /mnt/lustre' ');
[ 2548.610472] rc=0;
[ 2548.610472] if [ $running -eq 0 ] ; then
[ 2548.610472] mkdir -p /mnt/lustre;
[ 2548.610472] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/subsubdir_c0 /mnt/lustre;
[ 2548.610472] rc=$?;
[ 2548.610472] fi;
[ 2548.610472] exit $rc
[ 2548.654444] Lustre: Mounted lustre-client
[ 2548.655249] Lustre: Skipped 2 previous similar messages
[ 2548.660407] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2548.688564] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_subsubdir_c0
[ 2559.033204] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2559.033204] if [ $running -ne 0 ] ; then
[ 2559.033204] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2559.033204] lsof /mnt/lustre || need_kill=no;
[ 2559.033204] if [ x != x -a x$need_kill != xno ]; then
[ 2559.033204] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2559.033204] if
[ 2559.208559] Lustre: Unmounted lustre-client
[ 2559.209428] Lustre: Skipped 2 previous similar messages
[ 2559.216175] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2559.226603] Lustre: DEBUG MARKER:
[ 2559.226603] running=$(mount | grep -c /mnt/lustre' ');
[ 2559.226603] rc=0;
[ 2559.226603] if [ $running -eq 0 ] ; then
[ 2559.226603] mkdir -p /mnt/lustre;
[ 2559.226603] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 2559.226603] rc=$?;
[ 2559.226603] fi;
[ 2559.226603] exit $rc
[ 2559.273026] Lustre: Mounted lustre-client
[ 2559.279922] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2559.308839] Lustre: DEBUG MARKER: test -d /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[ 2566.925958] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[ 2567.644985] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[ 2596.429377] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2596.439949] Lustre: DEBUG MARKER:
[ 2596.439949] running=$(mount | grep -c /mnt/lustre' ');
[ 2596.439949] rc=0;
[ 2596.439949] if [ $running -eq 0 ] ; then
[ 2596.439949] mkdir -p /mnt/lustre;
[ 2596.439949] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 2596.439949] rc=$?;
[ 2596.439949] fi;
[ 2596.439949] exit $rc
[ 2596.455551] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2602.915587] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 27aa: test nodemap idmap range ======== 21:59:07 \(1734386347\)
[ 2603.245127] Lustre: DEBUG MARKER: == sanity-sec test 27aa: test nodemap idmap range ======== 21:59:07 (1734386347)
[ 2608.319168] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 27b: The new nodemap won\'t clear the old nodemap\'s fileset ========================================================== 21:59:12 \(1734386352\)
[ 2608.656977] Lustre: DEBUG MARKER: == sanity-sec test 27b: The new nodemap won't clear the old nodemap's fileset ========================================================== 21:59:12 (1734386352)
[ 2609.045180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_27b needs \>= 2 MDTs
[ 2609.382717] Lustre: DEBUG MARKER: SKIP: sanity-sec test_27b needs >= 2 MDTs
[ 2609.789093] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 28: check shared key rotation method == 21:59:13 \(1734386353\)
[ 2610.121788] Lustre: DEBUG MARKER: == sanity-sec test 28: check shared key rotation method == 21:59:13 (1734386353)
[ 2610.499448] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_28 need shared key feature for this test
[ 2610.829858] Lustre: DEBUG MARKER: SKIP: sanity-sec test_28 need shared key feature for this test
[ 2611.237657] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 29: check for missing shared key ====== 21:59:15 \(1734386355\)
[ 2611.568422] Lustre: DEBUG MARKER: == sanity-sec test 29: check for missing shared key ====== 21:59:15 (1734386355)
[ 2611.940230] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_29 need shared key feature for this test
[ 2612.271928] Lustre: DEBUG MARKER: SKIP: sanity-sec test_29 need shared key feature for this test
[ 2612.675161] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 30: check for invalid shared key ====== 21:59:16 \(1734386356\)
[ 2613.005899] Lustre: DEBUG MARKER: == sanity-sec test 30: check for invalid shared key ====== 21:59:16 (1734386356)
[ 2613.378583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_30 need shared key feature for this test
[ 2613.716264] Lustre: DEBUG MARKER: SKIP: sanity-sec test_30 need shared key feature for this test
[ 2614.123173] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 30b: basic test of all different SSK flavors ========================================================== 21:59:18 \(1734386358\)
[ 2614.453710] Lustre: DEBUG MARKER: == sanity-sec test 30b: basic test of all different SSK flavors ========================================================== 21:59:18 (1734386358)
[ 2614.825944] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_30b need shared key feature for this test
[ 2615.160267] Lustre: DEBUG MARKER: SKIP: sanity-sec test_30b need shared key feature for this test
[ 2615.570912] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 31: client mount option \'-o network\' == 21:59:19 \(1734386359\)
[ 2615.907564] Lustre: DEBUG MARKER: == sanity-sec test 31: client mount option '-o network' == 21:59:19 (1734386359)
[ 2616.572352] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show --net tcp
[ 2618.333103] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2618.343698] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2618.488646] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 2618.528385] Lustre: Unmounted lustre-client
[ 2621.574735] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2621.574735] if [ $running -ne 0 ] ; then
[ 2621.574735] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2621.574735] lsof /mnt/lustre || need_kill=no;
[ 2621.574735] if [ x != x -a x$need_kill != xno ]; then
[ 2621.574735] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2621.574735] if
[ 2622.953988] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 2622.953988] if [ $running -ne 0 ] ; then
[ 2622.953988] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 2622.953988] lsof /mnt/lustre2 || need_kill=no;
[ 2622.953988] if [ x != x -a x$need_kill != xno ]; then
[ 2622.953988] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 2622.953988]
[ 2657.011346] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[ 2657.011346] lctl dl | grep ' ST ' || true
[ 2657.364538] Key type lgssc unregistered
[ 2657.426919] LNet: 129735:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 2658.454942] LNet: Removed LNI 10.240.44.153@tcp
[ 2658.659067] Key type .llcrypt unregistered
[ 2658.659901] Key type ._llcrypt unregistered
[ 2665.068706] Key type ._llcrypt registered
[ 2665.069541] Key type .llcrypt registered
[ 2665.304370] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trevis-106vm14.trevis.whamcloud.com: executing load_modules
[ 2665.645224] Lustre: DEBUG MARKER: trevis-106vm14.trevis.whamcloud.com: executing load_modules
[ 2666.070324] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[ 2666.880891] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 2666.921475] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 2666.922660] LNet: Accept all, port 7988
[ 2668.450841] Key type lgssc registered
[ 2668.588253] Lustre: Echo OBD driver; http://www.lustre.org/
[ 2669.748231] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[ 2669.764266] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --if eth0 --net tcp999
[ 2669.772318] LNet: Added LNI 10.240.44.153@tcp999 [8/256/0/180]
[ 2676.748962] Autotest: Test running for 40 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2696.048201] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2696.048201] mpts=$(mount | grep -c /mnt/lustre' ');
[ 2696.048201] if [ $running -ne $mpts ]; then
[ 2696.048201] echo $(hostname) env are INSANE!;
[ 2696.048201] exit 1;
[ 2696.048201] fi
[ 2696.771305] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 2696.771305] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 2696.771305] if [ $running -ne $mpts ]; then
[ 2696.771305] echo $(hostname) env are INSANE!;
[ 2696.771305] exit 1;
[ 2696.771305] fi
[ 2761.816723] Key type lgssc unregistered
[ 2761.911786] LNet: 134411:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 2761.913433] LNet: Removed LNI 10.240.44.153@tcp
[ 2761.931177] LNet: Removed LNI 10.240.44.153@tcp999
[ 2762.123163] Key type .llcrypt unregistered
[ 2762.123970] Key type ._llcrypt unregistered
[ 2762.217207] Key type ._llcrypt registered
[ 2762.218009] Key type .llcrypt registered
[ 2762.265902] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[ 2763.076667] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 2763.114845] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 2763.116021] LNet: Accept all, port 7988
[ 2764.651089] Key type lgssc registered
[ 2764.803627] Lustre: Echo OBD driver; http://www.lustre.org/
[ 2764.859491] LNet: Added LNI 10.240.44.153@tcp999 [8/256/0/180]
[ 2764.961126] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2764.970894] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,network=tcp999 10.240.38.186@tcp999:/lustre /mnt/lustre
[ 2771.053006] Lustre: Mounted lustre-client
[ 2772.209009] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2772.219965] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2772.373854] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 2772.406100] Lustre: Unmounted lustre-client
[ 2774.956540] Key type lgssc unregistered
[ 2775.051876] LNet: 135104:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 2775.053519] LNet: Removed LNI 10.240.44.153@tcp
[ 2775.075175] LNet: Removed LNI 10.240.44.153@tcp999
[ 2775.263239] Key type .llcrypt unregistered
[ 2775.264067] Key type ._llcrypt unregistered
[ 2775.373085] Key type ._llcrypt registered
[ 2775.373908] Key type .llcrypt registered
[ 2775.419473] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[ 2776.219725] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 2776.255016] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 2776.256177] LNet: Accept all, port 7988
[ 2777.787105] Key type lgssc registered
[ 2777.930373] Lustre: Echo OBD driver; http://www.lustre.org/
[ 2777.975709] LNet: Added LNI 10.240.44.153@tcp999 [8/256/0/180]
[ 2778.077287] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2778.086993] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,network=tcp999 10.240.38.186@tcp999:/lustre /mnt/lustre
[ 2778.096845] LustreError: 135489:0:(obd_mount.c:1723:lmd_parse()) LNet Dynamic Peer Discovery is enabled on this node. 'network' mount option cannot be taken into account.
[ 2778.099385] LustreError: 135489:0:(obd_mount.c:1796:lmd_parse()) Bad mount options user_xattr,flock,
[ 2778.100921] LustreError: 135489:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -22
[ 2779.441314] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2779.636089] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2779.636089] if [ $running -ne 0 ] ; then
[ 2779.636089] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 2779.636089] lsof /mnt/lustre || need_kill=no;
[ 2779.636089] if [ x != x -a x$need_kill != xno ]; then
[ 2779.636089] pids=$(lsof -t /mnt/lustre | sort -u);
[ 2779.636089] if
[ 2780.266721] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 2780.266721] if [ $running -ne 0 ] ; then
[ 2780.266721] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 2780.266721] lsof /mnt/lustre2 || need_kill=no;
[ 2780.266721] if [ x != x -a x$need_kill != xno ]; then
[ 2780.266721] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 2780.266721]
[ 2813.992024] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[ 2813.992024] lctl dl | grep ' ST ' || true
[ 2814.368847] Key type lgssc unregistered
[ 2814.452082] LNet: 136493:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 2814.453737] LNet: Removed LNI 10.240.44.153@tcp
[ 2814.467325] LNet: Removed LNI 10.240.44.153@tcp999
[ 2814.667289] Key type .llcrypt unregistered
[ 2814.668105] Key type ._llcrypt unregistered
[ 2820.751523] Key type ._llcrypt registered
[ 2820.752408] Key type .llcrypt registered
[ 2820.798053] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[ 2821.599043] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 2821.640041] LNet: Added LNI 10.240.44.153@tcp [8/256/0/180]
[ 2821.641205] LNet: Accept all, port 7988
[ 2823.171215] Key type lgssc registered
[ 2823.321757] Lustre: Echo OBD driver; http://www.lustre.org/
[ 2848.600446] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 2848.600446] mpts=$(mount | grep -c /mnt/lustre' ');
[ 2848.600446] if [ $running -ne $mpts ]; then
[ 2848.600446] echo $(hostname) env are INSANE!;
[ 2848.600446] exit 1;
[ 2848.600446] fi
[ 2849.317141] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 2849.317141] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 2849.317141] if [ $running -ne $mpts ]; then
[ 2849.317141] echo $(hostname) env are INSANE!;
[ 2849.317141] exit 1;
[ 2849.317141] fi
[ 2907.044448] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2907.067133] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2908.113505] Lustre: Mounted lustre-client
[ 2908.314546] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2908.948695] Lustre: DEBUG MARKER:
[ 2908.948695] running=$(mount | grep -c /mnt/lustre' ');
[ 2908.948695] rc=0;
[ 2908.948695] if [ $running -eq 0 ] ; then
[ 2908.948695] mkdir -p /mnt/lustre;
[ 2908.948695] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 2908.948695] rc=$?;
[ 2908.948695] fi;
[ 2908.948695] exit $rc
[ 2920.643047] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 2925.184665] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 2925.516614] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 2925.894615] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 2926.714298] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 2926.742274] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 2926.756767] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 2927.114682] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 2928.128881] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 2929.142769] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[ 2934.768024] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 32: check for mgssec ================== 22:04:38 \(1734386678\)
[ 2935.103106] Lustre: DEBUG MARKER: == sanity-sec test 32: check for mgssec ================== 22:04:38 (1734386678)
[ 2935.904466] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_32 need shared key feature for this test
[ 2936.236602] Lustre: DEBUG MARKER: SKIP: sanity-sec test_32 need shared key feature for this test
[ 2936.634089] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 33: correct srpc flags for MGS connection ========================================================== 22:04:40 \(1734386680\)
[ 2936.968719] Lustre: DEBUG MARKER: == sanity-sec test 33: correct srpc flags for MGS connection ========================================================== 22:04:40 (1734386680)
[ 2937.346837] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_33 need shared key feature for this test
[ 2937.676609] Lustre: DEBUG MARKER: SKIP: sanity-sec test_33 need shared key feature for this test
[ 2938.077597] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 34: deny_unknown on default nodemap === 22:04:42 \(1734386682\)
[ 2938.411660] Lustre: DEBUG MARKER: == sanity-sec test 34: deny_unknown on default nodemap === 22:04:42 (1734386682)
[ 2943.847517] Lustre: lustre-OST0000-osc-ffff95090527a800: disconnect after 20s idle
[ 2952.973906] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 35: Check permissions when accessing changelogs ========================================================== 22:04:57 \(1734386697\)
[ 2953.303369] Lustre: DEBUG MARKER: == sanity-sec test 35: Check permissions when accessing changelogs ========================================================== 22:04:57 (1734386697)
[ 2955.727335] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 2976.515047] LustreError: 144163:0:(mdc_changelog.c:300:chlg_load()) lustre-MDT0000-mdc-ffff95090527a800: fail to open changelog catalog: rc = -13
[ 2977.830181] Autotest: Test running for 45 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2979.687602] Lustre: lustre-OST0006-osc-ffff95090527a800: disconnect after 25s idle
[ 2979.689047] Lustre: Skipped 6 previous similar messages
[ 2986.146387] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 36: control if clients can use encryption ========================================================== 22:05:30 \(1734386730\)
[ 2986.477736] Lustre: DEBUG MARKER: == sanity-sec test 36: control if clients can use encryption ========================================================== 22:05:30 (1734386730)
[ 2986.630449] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2986.641036] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2986.788250] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 2986.823846] Lustre: Unmounted lustre-client
[ 2986.832639] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2986.842940] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2986.886097] Lustre: Test dummy encryption mode enabled
[ 2986.887903] Lustre: Mounted lustre-client
[ 2986.916529] llcrypt: AES-256-CTS-CBC using implementation "cts-cbc-aes-aesni"
[ 2986.928230] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2986.939230] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2987.094612] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 2990.029486] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2990.039716] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2990.079896] Lustre: Test dummy encryption mode enabled
[ 2990.081609] Lustre: Mounted lustre-client
[ 2990.106255] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2990.117260] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2990.271064] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 2990.309173] Lustre: Unmounted lustre-client
[ 2990.310007] Lustre: Skipped 1 previous similar message
[ 2992.856045] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2992.866301] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 2992.912941] Lustre: lustre-MDT0000-mdc-ffff950905279800: connection denied by lustre-MDT0000_UUID: rc = -13
[ 2992.914703] LustreError: 145599:0:(lmv_obd.c:1434:lmv_statfs()) lustre-MDT0000-mdc-ffff950905279800: can't stat MDS #0: rc = -13
[ 2992.934203] Lustre: Unmounted lustre-client
[ 2992.936061] LustreError: 145599:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -13
[ 3006.899450] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3006.909258] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3006.948652] Lustre: Mounted lustre-client
[ 3006.983299] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3006.993621] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3007.152541] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3007.189878] Lustre: Unmounted lustre-client
[ 3007.198354] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3007.208126] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3007.419753] bash (144891): drop_caches: 3
[ 3007.730586] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 37: simple encrypted file ============= 22:05:51 \(1734386751\)
[ 3008.131149] Lustre: DEBUG MARKER: == sanity-sec test 37: simple encrypted file ============= 22:05:51 (1734386751)
[ 3008.284099] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3008.294405] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3008.461561] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3008.509335] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3008.519850] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3008.562398] Lustre: Test dummy encryption mode enabled
[ 3008.597975] llcrypt: AES-256-XTS using implementation "xts-aes-aesni"
[ 3011.785938] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3011.797065] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3011.948470] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3011.986572] Lustre: Unmounted lustre-client
[ 3011.987420] Lustre: Skipped 1 previous similar message
[ 3011.996608] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3012.006908] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3012.046488] Lustre: Mounted lustre-client
[ 3012.047323] Lustre: Skipped 2 previous similar messages
[ 3012.341697] bash (147063): drop_caches: 3
[ 3012.641312] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 38: encrypted file with hole ========== 22:05:56 \(1734386756\)
[ 3012.972753] Lustre: DEBUG MARKER: == sanity-sec test 38: encrypted file with hole ========== 22:05:56 (1734386756)
[ 3013.128584] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3013.138719] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3013.293668] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3013.335511] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3013.345642] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3013.386999] Lustre: Test dummy encryption mode enabled
[ 3014.853359] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3014.863975] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3015.015506] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3015.058803] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3015.068900] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3015.416853] bash (147718): drop_caches: 3
[ 3015.721660] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 39: rewrite data in already encrypted page ========================================================== 22:05:59 \(1734386759\)
[ 3016.045951] Lustre: DEBUG MARKER: == sanity-sec test 39: rewrite data in already encrypted page ========================================================== 22:05:59 (1734386759)
[ 3016.199139] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3016.209855] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3016.366022] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3016.411632] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3016.421624] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3017.894800] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3017.905122] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3018.061385] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3018.103171] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3018.112857] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3018.272691] bash (148339): drop_caches: 3
[ 3018.574427] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 40: exercise size of encrypted file === 22:06:02 \(1734386762\)
[ 3018.896009] Lustre: DEBUG MARKER: == sanity-sec test 40: exercise size of encrypted file === 22:06:02 (1734386762)
[ 3019.046442] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3019.056737] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3019.219073] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3019.267036] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3019.277077] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3019.317544] Lustre: Test dummy encryption mode enabled
[ 3019.318523] Lustre: Skipped 1 previous similar message
[ 3020.273197] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3020.292174] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3020.443589] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3020.481354] Lustre: Unmounted lustre-client
[ 3020.482207] Lustre: Skipped 5 previous similar messages
[ 3020.491955] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3020.502102] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3020.546244] Lustre: Mounted lustre-client
[ 3020.547048] Lustre: Skipped 5 previous similar messages
[ 3020.679433] bash (148955): drop_caches: 3
[ 3021.984536] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3021.994937] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3022.144727] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3022.189092] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3022.199076] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3022.368912] bash (148955): drop_caches: 3
[ 3022.664806] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 41: test race on encrypted file size \(1\) ========================================================== 22:06:06 \(1734386766\)
[ 3022.989878] Lustre: DEBUG MARKER: == sanity-sec test 41: test race on encrypted file size (1) ========================================================== 22:06:06 (1734386766)
[ 3023.141881] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3023.152676] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3023.313558] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3023.356408] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3023.366273] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3040.850883] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3040.861127] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3040.996736] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3041.030292] Lustre: Unmounted lustre-client
[ 3041.031093] Lustre: Skipped 2 previous similar messages
[ 3041.039523] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3041.049099] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3041.094153] Lustre: Mounted lustre-client
[ 3041.094942] Lustre: Skipped 2 previous similar messages
[ 3041.246599] bash (149755): drop_caches: 3
[ 3041.546113] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 42: test race on encrypted file size \(2\) ========================================================== 22:06:25 \(1734386785\)
[ 3041.870622] Lustre: DEBUG MARKER: == sanity-sec test 42: test race on encrypted file size (2) ========================================================== 22:06:25 (1734386785)
[ 3042.019487] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3042.029674] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3042.187618] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3042.238092] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3042.247865] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3042.285079] Lustre: Test dummy encryption mode enabled
[ 3042.286032] Lustre: Skipped 1 previous similar message
[ 3042.320580] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre2
[ 3042.330581] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre2
[ 3060.615412] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3060.626105] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3060.766461] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3060.811410] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3060.821777] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3060.892737] Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
[ 3060.903688] Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
[ 3061.059400] Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
[ 3061.207682] bash (150407): drop_caches: 3
[ 3061.507007] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 43: test race on encrypted file size \(3\) ========================================================== 22:06:45 \(1734386805\)
[ 3061.839632] Lustre: DEBUG MARKER: == sanity-sec test 43: test race on encrypted file size (3) ========================================================== 22:06:45 (1734386805)
[ 3061.998850] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3062.009738] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3062.175497] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3062.227931] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3062.238484] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3062.278526] Lustre: Test dummy encryption mode enabled
[ 3062.279529] Lustre: Skipped 1 previous similar message
[ 3062.315817] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre2
[ 3062.325946] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre2
[ 3080.090761] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3080.101400] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3080.248441] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3080.285479] Lustre: Unmounted lustre-client
[ 3080.286279] Lustre: Skipped 4 previous similar messages
[ 3080.294643] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3080.304903] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3080.343864] Lustre: Mounted lustre-client
[ 3080.344690] Lustre: Skipped 5 previous similar messages
[ 3080.376997] Lustre: DEBUG MARKER: grep -c /mnt/lustre2' ' /proc/mounts
[ 3080.387983] Lustre: DEBUG MARKER: lsof -t /mnt/lustre2
[ 3080.535302] Lustre: DEBUG MARKER: umount /mnt/lustre2 2>&1
[ 3080.751476] bash (151118): drop_caches: 3
[ 3081.057563] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 44: encrypted file access semantics: direct IO ========================================================== 22:07:05 \(1734386825\)
[ 3081.379679] Lustre: DEBUG MARKER: == sanity-sec test 44: encrypted file access semantics: direct IO ========================================================== 22:07:05 (1734386825)
[ 3081.536178] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3081.546703] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3081.705285] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3081.752958] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3081.763111] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3084.154122] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3084.164902] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3084.312502] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3084.357745] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3084.368045] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3084.586022] bash (151829): drop_caches: 3
[ 3084.889284] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 45: encrypted file access semantics: MMAP ========================================================== 22:07:09 \(1734386829\)
[ 3085.224908] Lustre: DEBUG MARKER: == sanity-sec test 45: encrypted file access semantics: MMAP ========================================================== 22:07:09 (1734386829)
[ 3085.375918] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3085.386891] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3085.554433] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3085.600936] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3085.611600] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3087.400929] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3087.411690] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3087.564956] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3087.611517] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3087.621738] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3087.902803] bash (152491): drop_caches: 3
[ 3088.212604] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 46: encrypted file access semantics without key ========================================================== 22:07:12 \(1734386832\)
[ 3088.540755] Lustre: DEBUG MARKER: == sanity-sec test 46: encrypted file access semantics without key ========================================================== 22:07:12 (1734386832)
[ 3088.699913] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3088.710971] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3088.871856] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3088.921512] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3088.932123] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3089.667930] bash (153108): drop_caches: 3
[ 3089.691750] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3089.705468] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3089.866882] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3089.920363] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3089.931203] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3090.097661] bash (153108): drop_caches: 3
[ 3090.190301] LustreError: 153411:0:(llite_lib.c:3949:ll_prep_md_op_data()) lustre: failed to setup filename: rc = -126
[ 3091.527729] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3091.538666] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3091.706977] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3091.754230] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3091.764498] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3091.975628] bash (153108): drop_caches: 3
[ 3092.280439] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 47: encrypted file access semantics: rename\/link ========================================================== 22:07:16 \(1734386836\)
[ 3092.609111] Lustre: DEBUG MARKER: == sanity-sec test 47: encrypted file access semantics: rename/link ========================================================== 22:07:16 (1734386836)
[ 3092.767906] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3092.778547] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3092.944373] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3092.990434] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3093.000931] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3094.275568] bash (153819): drop_caches: 3
[ 3094.298315] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3094.310454] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3094.477619] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3126.141724] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3126.152784] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3126.299876] bash (153819): drop_caches: 3
[ 3127.802402] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3127.813397] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3127.993980] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3128.039851] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3128.050697] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3128.225083] bash (153819): drop_caches: 3
[ 3128.527406] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 48a: encrypted file access semantics: truncate ========================================================== 22:07:52 \(1734386872\)
[ 3128.852808] Lustre: DEBUG MARKER: == sanity-sec test 48a: encrypted file access semantics: truncate ========================================================== 22:07:52 (1734386872)
[ 3129.008963] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3129.019482] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3129.174754] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3129.216939] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3129.227124] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3129.265720] Lustre: Test dummy encryption mode enabled
[ 3129.266680] Lustre: Skipped 5 previous similar messages
[ 3130.165911] bash (154714): drop_caches: 3
[ 3130.187775] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3130.199160] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3130.357002] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3130.403223] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3130.413430] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3130.594540] bash (154714): drop_caches: 3
[ 3131.897823] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3131.914712] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3132.074402] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3132.117208] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3132.127697] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3132.317447] bash (154714): drop_caches: 3
[ 3132.619363] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 48b: encrypted file: concurrent truncate ========================================================== 22:07:56 \(1734386876\)
[ 3132.944528] Lustre: DEBUG MARKER: == sanity-sec test 48b: encrypted file: concurrent truncate ========================================================== 22:07:56 (1734386876)
[ 3133.103985] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3133.114767] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3133.270181] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3133.314192] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3133.324517] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3172.932311] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3172.942984] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3173.092321] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3173.131431] Lustre: Unmounted lustre-client
[ 3173.132267] Lustre: Skipped 15 previous similar messages
[ 3173.141382] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3173.151246] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3173.187635] Lustre: Mounted lustre-client
[ 3173.188442] Lustre: Skipped 14 previous similar messages
[ 3173.349833] bash (155497): drop_caches: 3
[ 3173.651696] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 49: Avoid getxattr for encryption context ========================================================== 22:08:37 \(1734386917\)
[ 3173.973213] Lustre: DEBUG MARKER: == sanity-sec test 49: Avoid getxattr for encryption context ========================================================== 22:08:37 (1734386917)
[ 3174.131871] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3174.142737] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3174.303898] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3174.353731] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3174.363866] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3174.685376] bash (156245): drop_caches: 3
[ 3174.791804] bash (156245): drop_caches: 3
[ 3175.493522] bash (156245): drop_caches: 3
[ 3175.778669] bash (156245): drop_caches: 3
[ 3176.139108] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_49 2nd part needs \>= 2 MDTs
[ 3176.457420] Lustre: DEBUG MARKER: SKIP: sanity-sec test_49 2nd part needs >= 2 MDTs
[ 3177.867236] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3177.878003] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3178.022298] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3178.067483] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3178.077404] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3178.331250] bash (156245): drop_caches: 3
[ 3178.637128] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 50: DoM encrypted file ================ 22:08:42 \(1734386922\)
[ 3179.287892] Lustre: DEBUG MARKER: == sanity-sec test 50: DoM encrypted file ================ 22:08:42 (1734386922)
[ 3179.507121] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3179.517761] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3179.678559] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3179.728959] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3179.739053] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3180.062519] bash (157061): drop_caches: 3
[ 3180.157932] bash (157061): drop_caches: 3
[ 3180.254273] bash (157061): drop_caches: 3
[ 3180.315284] bash (157061): drop_caches: 3
[ 3180.382254] bash (157061): drop_caches: 3
[ 3180.449946] bash (157061): drop_caches: 3
[ 3180.524060] bash (157061): drop_caches: 3
[ 3180.591541] bash (157061): drop_caches: 3
[ 3180.658500] bash (157061): drop_caches: 3
[ 3180.725475] bash (157061): drop_caches: 3
[ 3180.795405] bash (157061): drop_caches: 3
[ 3180.858978] bash (157061): drop_caches: 3
[ 3180.925738] bash (157061): drop_caches: 3
[ 3180.988818] bash (157061): drop_caches: 3
[ 3181.055255] bash (157061): drop_caches: 3
[ 3181.113815] bash (157061): drop_caches: 3
[ 3181.240165] bash (157061): drop_caches: 3
[ 3181.297769] bash (157061): drop_caches: 3
[ 3181.381982] bash (157061): drop_caches: 3
[ 3181.448427] bash (157061): drop_caches: 3
[ 3181.523706] bash (157061): drop_caches: 3
[ 3181.590389] bash (157061): drop_caches: 3
[ 3181.665295] bash (157061): drop_caches: 3
[ 3181.732168] bash (157061): drop_caches: 3
[ 3181.807983] bash (157061): drop_caches: 3
[ 3181.865957] bash (157061): drop_caches: 3
[ 3183.193436] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3183.204367] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3183.354645] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3183.398823] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3183.409805] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3183.579690] bash (157061): drop_caches: 3
[ 3183.879060] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 51: FS capabilities =================== 22:08:48 \(1734386928\)
[ 3184.208746] Lustre: DEBUG MARKER: == sanity-sec test 51: FS capabilities =================== 22:08:48 (1734386928)
[ 3187.246980] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 52: Mirrored encrypted file =========== 22:08:51 \(1734386931\)
[ 3187.587214] Lustre: DEBUG MARKER: == sanity-sec test 52: Mirrored encrypted file =========== 22:08:51 (1734386931)
[ 3187.742114] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3187.752562] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3187.902865] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3188.029607] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3188.040476] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3188.687428] LustreError: 158838:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff95090a240800: inode [0x200002354:0x4:0x0] mdc close failed: rc = -22
[ 3189.998080] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3190.008968] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3190.179992] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3190.224884] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3190.234809] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3190.482434] bash (158580): drop_caches: 3
[ 3190.790053] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 53: Mixed PAGE_SIZE clients =========== 22:08:54 \(1734386934\)
[ 3191.114817] Lustre: DEBUG MARKER: == sanity-sec test 53: Mixed PAGE_SIZE clients =========== 22:08:54 (1734386934)
[ 3191.505349] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_53 Need 64K PAGE_SIZE client
[ 3191.840459] Lustre: DEBUG MARKER: SKIP: sanity-sec test_53 Need 64K PAGE_SIZE client
[ 3192.251513] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 54: Encryption policies with fscrypt == 22:08:56 \(1734386936\)
[ 3192.585289] Lustre: DEBUG MARKER: == sanity-sec test 54: Encryption policies with fscrypt == 22:08:56 (1734386936)
[ 3198.687151] Lustre: 159645:0:(policy.c:345:llcrypt_ioctl_set_policy()) inode 144115339825774598: forcing policy filenames_encryption_mode to null
[ 3198.728720] Lustre: 159656:0:(policy.c:473:llcrypt_ioctl_get_policy_ex()) inode 144115339825774598: returning policy filenames_encryption_mode as 4, but is in fact null
[ 3200.491337] Lustre: 159722:0:(policy.c:473:llcrypt_ioctl_get_policy_ex()) inode 144115339825774598: returning policy filenames_encryption_mode as 4, but is in fact null
[ 3200.493938] Lustre: 159722:0:(policy.c:473:llcrypt_ioctl_get_policy_ex()) Skipped 2 previous similar messages
[ 3207.297102] Lustre: 159912:0:(policy.c:473:llcrypt_ioctl_get_policy_ex()) inode 144115339825774598: returning policy filenames_encryption_mode as 4, but is in fact null
[ 3208.558159] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3208.568826] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3208.721120] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3240.769775] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3240.779863] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3240.827608] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 3240.839030] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//d54.sanity-sec
[ 3240.851958] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 3240.851958] umount /mnt/lustre
[ 3240.900487] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d54.sanity-sec /mnt/lustre
[ 3241.417149] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3242.433235] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3243.449145] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3245.061007] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3245.082377] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3245.244107] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3245.294043] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3245.304294] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3245.351808] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 3245.364387] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//d54.sanity-sec/vault
[ 3245.378497] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 3245.378497] umount /mnt/lustre
[ 3245.426992] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d54.sanity-sec/vault /mnt/lustre
[ 3246.853358] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3246.865130] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3247.020108] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3247.065515] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3247.076151] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3248.911376] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3249.927393] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3253.165841] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 55: access with seteuid =============== 22:09:57 \(1734386997\)
[ 3253.504391] Lustre: DEBUG MARKER: == sanity-sec test 55: access with seteuid =============== 22:09:57 (1734386997)
[ 3253.672059] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3253.682955] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3253.818501] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3264.488832] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3270.295282] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3270.305524] Lustre: DEBUG MARKER:
[ 3270.305524] running=$(mount | grep -c /mnt/lustre' ');
[ 3270.305524] rc=0;
[ 3270.305524] if [ $running -eq 0 ] ; then
[ 3270.305524] mkdir -p /mnt/lustre;
[ 3270.305524] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3270.305524] rc=$?;
[ 3270.305524] fi;
[ 3270.305524] exit $rc
[ 3270.357813] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3271.742176] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3271.752773] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3271.900626] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3278.710751] Autotest: Test running for 50 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3280.089416] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3280.099450] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3280.431078] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 56: FIEMAP on encrypted file ========== 22:10:24 \(1734387024\)
[ 3280.762644] Lustre: DEBUG MARKER: == sanity-sec test 56: FIEMAP on encrypted file ========== 22:10:24 (1734387024)
[ 3280.917965] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3280.928444] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3281.079263] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3281.124952] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3281.134949] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3281.174169] Lustre: Test dummy encryption mode enabled
[ 3281.175125] Lustre: Skipped 4 previous similar messages
[ 3282.778442] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3282.789226] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3282.939931] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3282.987891] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3282.997883] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3283.237535] bash (163265): drop_caches: 3
[ 3283.543185] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 57: security.c\/encryption.c xattr protection ========================================================== 22:10:27 \(1734387027\)
[ 3283.870109] Lustre: DEBUG MARKER: == sanity-sec test 57: security.c/encryption.c xattr protection ========================================================== 22:10:27 (1734387027)
[ 3284.169482] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3284.179929] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3284.335285] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3284.379359] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3284.389260] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3285.799515] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3285.810274] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3285.970098] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3286.021825] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3286.032046] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3286.253686] bash (163878): drop_caches: 3
[ 3286.566989] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 58: access to enc file\'s xattrs ======= 22:10:30 \(1734387030\)
[ 3286.894044] Lustre: DEBUG MARKER: == sanity-sec test 58: access to enc file's xattrs ======= 22:10:30 (1734387030)
[ 3287.053076] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3287.064740] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3287.221056] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3287.264852] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3287.274997] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3287.702006] bash (164525): drop_caches: 3
[ 3299.707786] bash (164525): drop_caches: 3
[ 3323.339359] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3323.350154] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3323.515834] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3323.577371] Lustre: Unmounted lustre-client
[ 3323.578190] Lustre: Skipped 18 previous similar messages
[ 3323.587596] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3323.598425] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3323.643682] Lustre: Mounted lustre-client
[ 3323.644490] Lustre: Skipped 18 previous similar messages
[ 3323.821724] bash (164525): drop_caches: 3
[ 3324.134445] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 59a: mirror resync of encrypted files without key ========================================================== 22:11:08 \(1734387068\)
[ 3324.453774] Lustre: DEBUG MARKER: == sanity-sec test 59a: mirror resync of encrypted files without key ========================================================== 22:11:08 (1734387068)
[ 3324.606469] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3324.617239] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3324.774827] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3324.827461] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3324.837802] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3325.335218] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3325.346103] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3325.498518] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3325.541210] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3325.551068] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3325.726509] bash (170161): drop_caches: 3
[ 3326.098310] bash (170161): drop_caches: 3
[ 3326.130325] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3326.142113] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3326.302003] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3326.347219] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3326.357782] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3327.840059] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3327.850842] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3328.004617] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3328.046366] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3328.056291] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3328.296723] bash (170161): drop_caches: 3
[ 3328.602977] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 59b: migrate\/extend\/split of encrypted files without key ========================================================== 22:11:12 \(1734387072\)
[ 3328.926342] Lustre: DEBUG MARKER: == sanity-sec test 59b: migrate/extend/split of encrypted files without key ========================================================== 22:11:12 (1734387072)
[ 3329.077219] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3329.087810] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3329.240380] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3329.284949] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3329.295366] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3329.589254] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3329.599966] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3329.750740] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3329.791740] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3329.801713] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3330.012782] bash (170947): drop_caches: 3
[ 3330.226031] bash (170947): drop_caches: 3
[ 3330.257189] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3330.270032] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3330.427944] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3330.473539] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3330.484088] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3330.582576] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3330.593545] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3330.742418] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3330.786482] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3330.796550] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3330.945282] bash (170947): drop_caches: 3
[ 3331.169481] bash (170947): drop_caches: 3
[ 3331.199235] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3331.211906] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3331.370690] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3331.417276] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3331.427578] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3331.633507] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3331.644450] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3331.800103] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3331.842752] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3331.852708] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3332.003818] bash (170947): drop_caches: 3
[ 3332.025744] LustreError: 171583:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff950909cd8800: inode [0x200002361:0x2:0x0] mdc close failed: rc = -22
[ 3332.111758] bash (170947): drop_caches: 3
[ 3332.142636] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3332.154132] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3332.318077] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3332.364066] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3332.377335] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3333.754437] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3333.765277] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3333.913740] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3333.956501] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3333.966411] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3334.163271] bash (170947): drop_caches: 3
[ 3334.495068] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 59c: MDT migrate of encrypted files without key ========================================================== 22:11:18 \(1734387078\)
[ 3334.816546] Lustre: DEBUG MARKER: == sanity-sec test 59c: MDT migrate of encrypted files without key ========================================================== 22:11:18 (1734387078)
[ 3335.205273] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_59c needs \>= 2 MDTs
[ 3335.538882] Lustre: DEBUG MARKER: SKIP: sanity-sec test_59c needs >= 2 MDTs
[ 3335.955415] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 60: Subdirmount of encrypted dir ====== 22:11:20 \(1734387080\)
[ 3336.298381] Lustre: DEBUG MARKER: == sanity-sec test 60: Subdirmount of encrypted dir ====== 22:11:20 (1734387080)
[ 3336.453834] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3336.464622] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3336.619631] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3336.670744] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3336.681241] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3337.005041] bash (172327): drop_caches: 3
[ 3337.029236] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3337.041666] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3337.198636] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3337.250468] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3337.261190] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3337.303562] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 3337.316847] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//d60.sanity-sec
[ 3337.329319] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 3337.329319] umount /mnt/lustre
[ 3337.369826] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre/d60.sanity-sec /mnt/lustre
[ 3337.582513] bash (172327): drop_caches: 3
[ 3337.616101] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3337.628206] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3337.789778] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3337.833130] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3337.843786] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3337.890254] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[ 3337.902071] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//d60.sanity-sec
[ 3337.914536] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[ 3337.914536] umount /mnt/lustre
[ 3337.957187] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre/d60.sanity-sec /mnt/lustre
[ 3339.408757] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3339.419884] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3339.577623] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3339.625221] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3339.635653] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3339.837319] bash (172327): drop_caches: 3
[ 3340.147345] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 61: Nodemap enforces read-only mount == 22:11:24 \(1734387084\)
[ 3340.472396] Lustre: DEBUG MARKER: == sanity-sec test 61: Nodemap enforces read-only mount == 22:11:24 (1734387084)
[ 3340.958428] Lustre: DEBUG MARKER: lctl get_param -n at_max
[ 3343.037456] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3343.048671] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3343.203620] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3353.589758] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3360.132913] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3360.143748] Lustre: DEBUG MARKER:
[ 3360.143748] running=$(mount | grep -c /mnt/lustre' ');
[ 3360.143748] rc=0;
[ 3360.143748] if [ $running -eq 0 ] ; then
[ 3360.143748] mkdir -p /mnt/lustre;
[ 3360.143748] mount -t lustre -o user_xattr,flock,rw 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3360.143748] rc=$?;
[ 3360.143748] fi;
[ 3360.143748] exit $rc
[ 3360.200172] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3360.256248] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3360.267056] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3360.420892] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3363.247036] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3363.258550] Lustre: DEBUG MARKER:
[ 3363.258550] running=$(mount | grep -c /mnt/lustre' ');
[ 3363.258550] rc=0;
[ 3363.258550] if [ $running -eq 0 ] ; then
[ 3363.258550] mkdir -p /mnt/lustre;
[ 3363.258550] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3363.258550] rc=$?;
[ 3363.258550] fi;
[ 3363.258550] exit $rc
[ 3363.303499] Lustre: lustre-MDT0000-mdc-ffff950909d61000: connection denied by lustre-MDT0000_UUID: rc = -30
[ 3363.305279] LustreError: 174901:0:(lmv_obd.c:1434:lmv_statfs()) lustre-MDT0000-mdc-ffff950909d61000: can't stat MDS #0: rc = -30
[ 3363.307253] LustreError: 174901:0:(llite_lib.c:471:client_common_fill_super()) lustre-clilmv-ffff950909d61000: mount failed with -30, forcing read-only mount.
[ 3363.324236] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3363.364518] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3363.375619] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3363.534297] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3363.577230] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3363.587733] Lustre: DEBUG MARKER:
[ 3363.587733] running=$(mount | grep -c /mnt/lustre' ');
[ 3363.587733] rc=0;
[ 3363.587733] if [ $running -eq 0 ] ; then
[ 3363.587733] mkdir -p /mnt/lustre;
[ 3363.587733] mount -t lustre -o user_xattr,flock,rw 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3363.587733] rc=$?;
[ 3363.587733] fi;
[ 3363.587733] exit $rc
[ 3363.648238] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3363.689147] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3363.701048] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3363.852854] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3363.895760] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3363.906473] Lustre: DEBUG MARKER:
[ 3363.906473] running=$(mount | grep -c /mnt/lustre' ');
[ 3363.906473] rc=0;
[ 3363.906473] if [ $running -eq 0 ] ; then
[ 3363.906473] mkdir -p /mnt/lustre;
[ 3363.906473] mount -t lustre -o user_xattr,flock,ro 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3363.906473] rc=$?;
[ 3363.906473] fi;
[ 3363.906473] exit $rc
[ 3363.960113] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3363.997901] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3364.009123] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3364.163410] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3364.208512] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3364.219530] Lustre: DEBUG MARKER:
[ 3364.219530] running=$(mount | grep -c /mnt/lustre' ');
[ 3364.219530] rc=0;
[ 3364.219530] if [ $running -eq 0 ] ; then
[ 3364.219530] mkdir -p /mnt/lustre;
[ 3364.219530] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3364.219530] rc=$?;
[ 3364.219530] fi;
[ 3364.219530] exit $rc
[ 3364.264211] LustreError: 175142:0:(llite_lib.c:471:client_common_fill_super()) lustre-clilmv-ffff95090fad4000: mount failed with -30, forcing read-only mount.
[ 3364.266618] LustreError: 175142:0:(llite_lib.c:471:client_common_fill_super()) Skipped 1 previous similar message
[ 3364.283254] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3364.325807] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3364.336964] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3364.486949] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3373.993167] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3374.003888] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3374.345438] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 62: e2fsck with encrypted files ======= 22:11:58 \(1734387118\)
[ 3374.691831] Lustre: DEBUG MARKER: == sanity-sec test 62: e2fsck with encrypted files ======= 22:11:58 (1734387118)
[ 3374.857188] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3374.868374] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3375.028464] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3375.077305] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3375.087674] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3375.723469] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 3375.723469] if [ $running -ne 0 ] ; then
[ 3375.723469] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 3375.723469] lsof /mnt/lustre || need_kill=no;
[ 3375.723469] if [ x != x -a x$need_kill != xno ]; then
[ 3375.723469] pids=$(lsof -t /mnt/lustre | sort -u);
[ 3375.723469] if
[ 3377.104304] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 3377.104304] if [ $running -ne 0 ] ; then
[ 3377.104304] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 3377.104304] lsof /mnt/lustre2 || need_kill=no;
[ 3377.104304] if [ x != x -a x$need_kill != xno ]; then
[ 3377.104304] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 3377.104304]
[ 3413.014566] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 3413.014566] mpts=$(mount | grep -c /mnt/lustre' ');
[ 3413.014566] if [ $running -ne $mpts ]; then
[ 3413.014566] echo $(hostname) env are INSANE!;
[ 3413.014566] exit 1;
[ 3413.014566] fi
[ 3413.751485] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 3413.751485] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 3413.751485] if [ $running -ne $mpts ]; then
[ 3413.751485] echo $(hostname) env are INSANE!;
[ 3413.751485] exit 1;
[ 3413.751485] fi
[ 3472.835889] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3472.846582] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3473.100215] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3473.733748] Lustre: DEBUG MARKER:
[ 3473.733748] running=$(mount | grep -c /mnt/lustre' ');
[ 3473.733748] rc=0;
[ 3473.733748] if [ $running -eq 0 ] ; then
[ 3473.733748] mkdir -p /mnt/lustre;
[ 3473.733748] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3473.733748] rc=$?;
[ 3473.733748] fi;
[ 3473.733748] exit $rc
[ 3474.492831] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3479.498056] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 3479.832295] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 3480.216882] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 3481.042525] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 3487.732370] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3487.743453] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3487.888163] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3487.934760] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3487.945817] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3488.162949] bash (176178): drop_caches: 3
[ 3488.471818] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 63: fid2path with encrypted files ===== 22:13:52 \(1734387232\)
[ 3488.797224] Lustre: DEBUG MARKER: == sanity-sec test 63: fid2path with encrypted files ===== 22:13:52 (1734387232)
[ 3493.653374] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3494.669387] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3495.685127] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3497.767913] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3498.784938] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3499.800036] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n llite.*.enable_filename_encryption | head -n1
[ 3501.457761] Lustre: 180714:0:(policy.c:345:llcrypt_ioctl_set_policy()) inode 144115373044662287: forcing policy filenames_encryption_mode to null
[ 3501.574933] Lustre: 180744:0:(policy.c:473:llcrypt_ioctl_get_policy_ex()) inode 144115373044662287: returning policy filenames_encryption_mode as 4, but is in fact null
[ 3501.577482] Lustre: 180744:0:(policy.c:473:llcrypt_ioctl_get_policy_ex()) Skipped 1 previous similar message
[ 3503.261054] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 64a: Nodemap enforces file_perms RBAC roles ========================================================== 22:14:07 \(1734387247\)
[ 3503.597636] Lustre: DEBUG MARKER: == sanity-sec test 64a: Nodemap enforces file_perms RBAC roles ========================================================== 22:14:07 (1734387247)
[ 3515.955813] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3537.200558] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 64b: Nodemap enforces dne_ops RBAC roles ========================================================== 22:14:41 \(1734387281\)
[ 3537.542850] Lustre: DEBUG MARKER: == sanity-sec test 64b: Nodemap enforces dne_ops RBAC roles ========================================================== 22:14:41 (1734387281)
[ 3537.936068] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_64b mdt count 1, skipping dne_ops role
[ 3538.278629] Lustre: DEBUG MARKER: SKIP: sanity-sec test_64b mdt count 1, skipping dne_ops role
[ 3538.698333] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 64c: Nodemap enforces quota_ops RBAC roles ========================================================== 22:14:42 \(1734387282\)
[ 3539.035815] Lustre: DEBUG MARKER: == sanity-sec test 64c: Nodemap enforces quota_ops RBAC roles ========================================================== 22:14:42 (1734387282)
[ 3552.704257] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3563.768397] LustreError: 185748:0:(mdc_request.c:2134:mdc_quotactl()) lustre-MDT0000-mdc-ffff950909d60000: ptlrpc_queue_wait failed: rc = -1
[ 3573.491244] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 64d: Nodemap enforces byfid_ops RBAC roles ========================================================== 22:15:17 \(1734387317\)
[ 3573.829735] Lustre: DEBUG MARKER: == sanity-sec test 64d: Nodemap enforces byfid_ops RBAC roles ========================================================== 22:15:17 (1734387317)
[ 3579.565194] Autotest: Test running for 55 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3584.307923] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3607.105355] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 64e: Nodemap enforces chlg_ops RBAC roles ========================================================== 22:15:51 \(1734387351\)
[ 3607.442760] Lustre: DEBUG MARKER: == sanity-sec test 64e: Nodemap enforces chlg_ops RBAC roles ========================================================== 22:15:51 (1734387351)
[ 3619.622636] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3632.554652] LustreError: 191341:0:(mdc_changelog.c:300:chlg_load()) lustre-MDT0000-mdc-ffff950909d60000: fail to open changelog catalog: rc = -13
[ 3646.036813] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 64f: Nodemap enforces fscrypt_admin RBAC roles ========================================================== 22:16:30 \(1734387390\)
[ 3646.375963] Lustre: DEBUG MARKER: == sanity-sec test 64f: Nodemap enforces fscrypt_admin RBAC roles ========================================================== 22:16:30 (1734387390)
[ 3656.903809] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3669.444918] Lustre: 194028:0:(policy.c:345:llcrypt_ioctl_set_policy()) inode 144115373044662314: forcing policy filenames_encryption_mode to null
[ 3669.475315] Lustre: 194031:0:(policy.c:473:llcrypt_ioctl_get_policy_ex()) inode 144115373044662314: returning policy filenames_encryption_mode as 4, but is in fact null
[ 3690.650171] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 64g: Nodemap enforces server_upcall RBAC role ========================================================== 22:17:14 \(1734387434\)
[ 3690.989174] Lustre: DEBUG MARKER: == sanity-sec test 64g: Nodemap enforces server_upcall RBAC role ========================================================== 22:17:14 (1734387434)
[ 3701.828565] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[ 3748.964563] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 65: lfs find -printf %La and --attrs support ========================================================== 22:18:13 \(1734387493\)
[ 3749.390887] Lustre: DEBUG MARKER: == sanity-sec test 65: lfs find -printf %La and --attrs support ========================================================== 22:18:13 (1734387493)
[ 3767.761401] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3767.773096] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3767.929311] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3767.977074] Lustre: Unmounted lustre-client
[ 3767.977921] Lustre: Skipped 27 previous similar messages
[ 3767.987479] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3767.997837] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3768.048191] Lustre: Test dummy encryption mode enabled
[ 3768.049140] Lustre: Skipped 12 previous similar messages
[ 3768.052262] Lustre: Mounted lustre-client
[ 3768.053041] Lustre: Skipped 27 previous similar messages
[ 3769.952380] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3769.963611] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3770.117844] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3770.172045] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3770.182323] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3770.493191] bash (198352): drop_caches: 3
[ 3770.847954] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 68: all config logs are processed ===== 22:18:35 \(1734387515\)
[ 3771.175829] Lustre: DEBUG MARKER: == sanity-sec test 68: all config logs are processed ===== 22:18:35 (1734387515)
[ 3771.317232] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3771.328970] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3771.490791] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3771.546989] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3771.557802] Lustre: DEBUG MARKER:
[ 3771.557802] running=$(mount | grep -c /mnt/lustre' ');
[ 3771.557802] rc=0;
[ 3771.557802] if [ $running -eq 0 ] ; then
[ 3771.557802] mkdir -p /mnt/lustre;
[ 3771.557802] mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre;
[ 3771.557802] rc=$?;
[ 3771.557802] fi;
[ 3771.557802] exit $rc
[ 3771.577894] LustreError: 136828:0:(niobuf.c:737:ptl_send_rpc()) cfs_fail_timeout id 51d sleeping for 20000ms
[ 3777.705570] LustreError: 199230:0:(mgc_request.c:232:do_config_log_add()) MGC10.240.38.186@tcp: failed processing log, type 1: rc = -5
[ 3791.649522] LustreError: 136828:0:(niobuf.c:737:ptl_send_rpc()) cfs_fail_timeout id 51d awake
[ 3794.481020] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 3794.515592] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 3794.527627] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 3794.690432] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1
[ 3796.088216] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 3796.098884] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.38.186@tcp:/lustre /mnt/lustre
[ 3796.440891] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 69: check upcall incorrect values ===== 22:19:00 \(1734387540\)
[ 3796.777817] Lustre: DEBUG MARKER: == sanity-sec test 69: check upcall incorrect values ===== 22:19:00 (1734387540)
[ 3800.554275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 70: targets have local copy of sptlrpc llog ========================================================== 22:19:04 \(1734387544\)
[ 3800.889906] Lustre: DEBUG MARKER: == sanity-sec test 70: targets have local copy of sptlrpc llog ========================================================== 22:19:04 (1734387544)
[ 3801.282888] Lustre: DEBUG MARKER: /usr/sbin/lctl mark SKIP: sanity-sec test_70 need shared key feature for this test
[ 3801.615184] Lustre: DEBUG MARKER: SKIP: sanity-sec test_70 need shared key feature for this test
[ 3802.043552] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 71: encryption does not remove project flag ========================================================== 22:19:06 \(1734387546\)
[ 3802.379361] Lustre: DEBUG MARKER: == sanity-sec test 71: encryption does not remove project flag ========================================================== 22:19:06 (1734387546)
[ 3806.573878] Lustre: 200591:0:(policy.c:345:llcrypt_ioctl_set_policy()) inode 144115373078216711: forcing policy filenames_encryption_mode to null
[ 3808.226699] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 72: dynamic nodemap properties ======== 22:19:12 \(1734387552\)
[ 3808.570483] Lustre: DEBUG MARKER: == sanity-sec test 72: dynamic nodemap properties ======== 22:19:12 (1734387552)
[ 3819.567410] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cleanup: ======================================================
[ 3819.899040] Lustre: DEBUG MARKER: cleanup: ======================================================
[ 3820.299507] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test complete, duration 3175 sec =========== 22:19:24 \(1734387564\)
[ 3820.649881] Lustre: DEBUG MARKER: == sanity-sec test complete, duration 3175 sec =========== 22:19:24 (1734387564)
[ 3821.051483] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: start cleanup 22:19:25 \(1734387565\) ===
[ 3821.391816] Lustre: DEBUG MARKER: === sanity-sec: start cleanup 22:19:25 (1734387565) ===
[ 3822.353595] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: finish cleanup 22:19:26 \(1734387566\) ===
[ 3822.682153] Lustre: DEBUG MARKER: === sanity-sec: finish cleanup 22:19:26 (1734387566) ===
[ 3823.019269] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 3823.019269] if [ $running -ne 0 ] ; then
[ 3823.019269] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 3823.019269] lsof /mnt/lustre || need_kill=no;
[ 3823.019269] if [ x != x -a x$need_kill != xno ]; then
[ 3823.019269] pids=$(lsof -t /mnt/lustre | sort -u);
[ 3823.019269] if
[ 3824.442618] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 3824.442618] if [ $running -ne 0 ] ; then
[ 3824.442618] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 3824.442618] lsof /mnt/lustre2 || need_kill=no;
[ 3824.442618] if [ x != x -a x$need_kill != xno ]; then
[ 3824.442618] pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 3824.442618]
[ 3859.179584] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1 || true
[ 3860.650299] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[ 3860.650299] lctl dl | grep ' ST ' || true
[ 3861.020207] Key type lgssc unregistered
[ 3861.093840] LNet: 203615:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 3861.109793] LNet: Removed LNI 10.240.44.153@tcp
[ 3861.338417] Key type .llcrypt unregistered
[ 3861.339322] Key type ._llcrypt unregistered
[ 3885.718082] Autotest: Test step complete (lustre-reviews_review-ldiskfs-ubuntu_109661.43)