Initializing cgroup subsys cpuset Initializing cgroup subsys cpu Linux version 2.6.32.masterDebugA (build@build63) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #2 SMP Wed Sep 5 12:25:38 PDT 2012 Command line: ro root=/dev/mapper/vg_mdt-lv_root nomodeset rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_mdt/lv_root rd_NO_MD crashkernel=auto rd_LVM_LV=vg_mdt/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM console=tty0 console=ttyS0,9800n8 SYSFONT=latarcyrheb-sun16 noapic KERNEL supported cpus: Intel GenuineIntel AMD AuthenticAMD Centaur CentaurHauls BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 000000003fff0000 (usable) BIOS-e820: 000000003fff0000 - 0000000040000000 (ACPI data) BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved) DMI 2.5 present. SMBIOS version 2.5 @ 0xFFF30 last_pfn = 0x3fff0 max_arch_pfn = 0x400000000 x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 CPU MTRRs all blank - virtualized system. init_memory_mapping: 0000000000000000-000000003fff0000 RAMDISK: 37ac1000 - 37fefade ACPI: RSDP 00000000000e0000 00024 (v02 VBOX ) ACPI: XSDT 000000003fff0030 0003C (v01 VBOX VBOXXSDT 00000001 ASL 00000061) ACPI: FACP 000000003fff00f0 000F4 (v04 VBOX VBOXFACP 00000001 ASL 00000061) ACPI: DSDT 000000003fff0470 01B96 (v01 VBOX VBOXBIOS 00000002 INTL 20100528) ACPI: FACS 000000003fff0200 00040 ACPI: APIC 000000003fff0240 00054 (v02 VBOX VBOXAPIC 00000001 ASL 00000061) ACPI: SSDT 000000003fff02a0 001CC (v01 VBOX VBOXCPUT 00000002 INTL 20100528) No NUMA configuration found Faking a node at 0000000000000000-000000003fff0000 Bootmem setup node 0 0000000000000000-000000003fff0000 NODE_DATA [0000000000009000 - 000000000003cfff] bootmap [000000000003d000 - 0000000000044fff] pages 8 (7 early reservations) ==> bootmem [0000000000 - 003fff0000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000] #1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000] #2 [0001000000 - 0002012024] TEXT DATA BSS ==> [0001000000 - 0002012024] #3 [0037ac1000 - 0037fefade] RAMDISK ==> [0037ac1000 - 0037fefade] #4 [000009fc00 - 0000100000] BIOS reserved ==> [000009fc00 - 0000100000] #5 [0002013000 - 0002013079] BRK ==> [0002013000 - 0002013079] #6 [0000008000 - 0000009000] PGTABLE ==> [0000008000 - 0000009000] found SMP MP-table at [ffff88000009fff0] 9fff0 Zone PFN ranges: DMA 0x00000001 -> 0x00001000 DMA32 0x00001000 -> 0x00100000 Normal 0x00100000 -> 0x00100000 Movable zone start PFN for each node early_node_map[2] active PFN ranges 0: 0x00000001 -> 0x0000009f 0: 0x00000100 -> 0x0003fff0 ACPI: PM-Timer IO Port: 0x4008 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) ACPI: Skipping IOAPIC probe due to 'noapic' option. Using ACPI for processor (LAPIC) configuration information Intel MultiProcessor Specification v1.4 MPTABLE: OEM ID: VBOXCPU MPTABLE: Product ID: VirtualBox MPTABLE: APIC at: 0xFEE00000 I/O APIC #0 Version 17 at 0xFEC00000. Processors: 1 SMP: Allowing 1 CPUs, 0 hotplug CPUs PM: Registered nosave memory: 000000000009f000 - 00000000000a0000 PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000 PM: Registered nosave memory: 00000000000f0000 - 0000000000100000 Allocating PCI resources starting at 40000000 (gap: 40000000:bffc0000) Booting paravirtualized kernel on bare hardware NR_CPUS:4096 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 PERCPU: Embedded 31 pages/cpu @ffff880002200000 s94424 r8192 d24360 u2097152 pcpu-alloc: s94424 r8192 d24360 u2097152 alloc=1*2097152 pcpu-alloc: [0] 0 Built 1 zonelists in Node order, mobility grouping on. Total pages: 258346 Policy zone: DMA32 Kernel command line: ro root=/dev/mapper/vg_mdt-lv_root nomodeset rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_mdt/lv_root rd_NO_MD rd_LVM_LV=vg_mdt/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM console=tty0 console=ttyS0,9800n8 SYSFONT=latarcyrheb-sun16 noapic PID hash table entries: 4096 (order: 3, 32768 bytes) Checking aperture... No AGP bridge found Memory: 1011388k/1048512k available (5160k kernel code, 392k absent, 36732k reserved, 7158k data, 1260k init) Hierarchical RCU implementation. NR_IRQS:33024 nr_irqs:256 Console: colour VGA+ 80x25 console [tty0] enabled console [ttyS0] enabled allocated 8388608 bytes of page_cgroup please try 'cgroup_disable=memory' option if you don't want memory cgroups Fast TSC calibration using PIT Detected 2285.652 MHz processor. Calibrating delay loop (skipped), value calculated using timer frequency.. 4571.30 BogoMIPS (lpj=2285652) pid_max: default: 32768 minimum: 301 Security Framework initialized SELinux: Initializing. Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) Mount-cache hash table entries: 256 Initializing cgroup subsys ns Initializing cgroup subsys cpuacct Initializing cgroup subsys memory Initializing cgroup subsys devices Initializing cgroup subsys freezer Initializing cgroup subsys net_cls Initializing cgroup subsys blkio Initializing cgroup subsys perf_event Initializing cgroup subsys net_prio mce: CPU supports 0 MCE banks using mwait in idle threads. SMP alternatives: switching to UP code Freeing SMP alternatives: 34k freed ACPI: Core revision 20090903 ACPI: setting ELCR to 0200 (from 0e20) ftrace: converting mcount calls to 0f 1f 44 00 00 ftrace: allocating 21034 entries in 83 pages Setting APIC routing to flat CPU0: Intel(R) Core(TM) i7-3610QM CPU @ 2.30GHz stepping 09 Performance Events: unsupported p6 CPU model 58 no PMU driver, software events only. NMI watchdog disabled (cpu0): hardware events not enabled Brought up 1 CPUs Total of 1 processors activated (4571.30 BogoMIPS). devtmpfs: initialized regulator: core version 0.5 NET: Registered protocol family 16 ACPI: bus type pci registered PCI: Using configuration type 1 for base access bio: create slab at 0 ACPI: Executed 1 blocks of module-level executable AML code ACPI: Interpreter enabled ACPI: (supports S0 S5) ACPI: Using PIC for interrupt routing ACPI: No dock devices found. HEST: Table not found. PCI: Ignoring host bridge windows from ACPI; if necessary, use "pci=use_crs" and report a bug ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) ACPI: PCI Interrupt Link [LNKA] (IRQs *5 9 10 11) ACPI: PCI Interrupt Link [LNKB] (IRQs 5 9 10 *11) ACPI: PCI Interrupt Link [LNKC] (IRQs 5 9 *10 11) ACPI: PCI Interrupt Link [LNKD] (IRQs 5 *9 10 11) vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none vgaarb: loaded vgaarb: bridge control possible 0000:00:02.0 SCSI subsystem initialized usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb PCI: Using ACPI for IRQ routing NetLabel: Initializing NetLabel: domain hash size = 128 NetLabel: protocols = UNLABELED CIPSOv4 NetLabel: unlabeled traffic allowed by default Switching to clocksource jiffies pnp: PnP ACPI init ACPI: bus type pnp registered pnp: PnP ACPI: found 6 devices ACPI: ACPI bus type pnp unregistered Switching to clocksource acpi_pm NET: Registered protocol family 2 IP route cache hash table entries: 32768 (order: 6, 262144 bytes) TCP established hash table entries: 131072 (order: 9, 2097152 bytes) TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) TCP: Hash tables configured (established 131072 bind 65536) TCP reno registered NET: Registered protocol family 1 pci 0000:00:00.0: Limiting direct PCI/PCI transfers pci 0000:00:01.0: Activating ISA DMA hang workarounds ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11 pci 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, low) -> IRQ 11 pci 0000:00:06.0: PCI INT A disabled Trying to unpack rootfs image as initramfs... Freeing initrd memory: 5306k freed platform rtc_cmos: registered platform RTC device (no PNP device found) audit: initializing netlink socket (disabled) type=2000 audit(1346991072.299:1): initialized HugeTLB registered 2 MB page size, pre-allocated 0 pages VFS: Disk quotas dquot_6.5.2 Dquot-cache hash table entries: 512 (order 0, 4096 bytes) msgmni has been set to 1985 alg: No test for stdrng (krng) ksign: Installing public key data Loading keyring - Added public key 62EEDDAA861A809E - User ID: Red Hat, Inc. (Kernel Module GPG key) - Added public key D4A26C9CCD09BEDA - User ID: Red Hat Enterprise Linux Driver Update Program Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252) io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered (default) io scheduler cfq registered pci_hotplug: PCI Hot Plug PCI Core version: 0.5 pciehp: PCI Express Hot Plug Controller Driver version: 0.4 acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 ACPI: AC Adapter [AC] (on-line) input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 ACPI: Power Button [PWRF] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1 ACPI: Sleep Button [SLPF] ERST: Table is not found! GHES: HEST is not enabled! ACPI: Battery Slot [BAT0] (battery present) Non-volatile memory driver v1.3 Linux agpgart interface v0.103 crash memory driver: version 1.1 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled ˙serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A 00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A brd: module loaded loop: module loaded input: Macintosh mouse button emulation as /devices/virtual/input/input2 Fixed MDIO Bus: probed ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver ohci_hcd 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, low) -> IRQ 11 ohci_hcd 0000:00:06.0: OHCI Host Controller ohci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 1 ohci_hcd 0000:00:06.0: irq 11, io mem 0xf0804000 usb usb1: New USB device found, idVendor=1d6b, idProduct=0001 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: OHCI Host Controller usb usb1: Manufacturer: Linux 2.6.32.masterDebugA ohci_hcd usb usb1: SerialNumber: 0000:00:06.0 usb usb1: configuration #1 chosen from 1 choice hub 1-0:1.0: USB hub found hub 1-0:1.0: 8 ports detected uhci_hcd: USB Universal Host Controller Interface driver PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12 serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX port at 0x60,0x64 irq 12 mice: PS/2 mouse device common for all mice input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3 rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 rtc0: alarms up to one day, 114 bytes nvram cpuidle: using governor ladder cpuidle: using governor menu EFI Variables Facility v0.08 2004-May-17 usbcore: registered new interface driver hiddev usbcore: registered new interface driver usbhid usbhid: v2.6:USB HID core driver input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input4 TCP cubic registered Initializing XFRM netlink socket NET: Registered protocol family 17 registered taskstats version 1 rtc_cmos rtc_cmos: setting system clock to 2012-09-07 04:11:13 UTC (1346991073) Initalizing network drop monitor service Freeing unused kernel memory: 1260k freed Write protecting the kernel read-only data: 10240k Freeing unused kernel memory: 964k freed Freeing unused kernel memory: 1728k freed dracut: dracut-004-256.el6 dracut: rd_NO_LUKS: removing cryptoluks activation device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.22.6-ioctl (2011-10-19) initialised: dm-devel@redhat.com udev: starting version 147 dracut: Starting plymouth daemon %G%GACPI: PCI Interrupt Link [LNKA] enabled at IRQ 5 ahci 0000:00:0d.0: PCI INT A -> Link[LNKA] -> GSI 5 (level, low) -> IRQ 5 ahci: SSS flag set, parallel bus scan disabled ahci 0000:00:0d.0: AHCI 0001.0100 32 slots 2 ports 3 Gbps 0x3 impl SATA mode ahci 0000:00:0d.0: flags: 64bit ncq stag only ccc scsi0 : ahci scsi1 : ahci ata1: SATA max UDMA/133 abar m8192@0xf0840000 port 0xf0840100 irq 5 ata2: SATA max UDMA/133 abar m8192@0xf0840000 port 0xf0840180 irq 5 Refined TSC clocksource calibration: 2285.435 MHz. Switching to clocksource tsc ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata1.00: ATA-6: VBOX HARDDISK, 1.0, max UDMA/133 ata1.00: 33296384 sectors, multi 128: LBA48 NCQ (depth 31/32) ata1.00: configured for UDMA/133 scsi 0:0:0:0: Direct-Access ATA VBOX HARDDISK 1.0 PQ: 0 ANSI: 5 ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata2.00: ATA-6: VBOX HARDDISK, 1.0, max UDMA/133 ata2.00: 16777216 sectors, multi 128: LBA48 NCQ (depth 31/32) ata2.00: configured for UDMA/133 scsi 1:0:0:0: Direct-Access ATA VBOX HARDDISK 1.0 PQ: 0 ANSI: 5 scsi2 : ata_piix scsi3 : ata_piix ata3: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6 bmdma 0xd000 irq 14 ata4: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0xd008 irq 15 ata4.00: ATAPI: VBOX CD-ROM, 1.0, max UDMA/133 ata3.00: ATAPI: VBOX CD-ROM, 1.0, max UDMA/133 ata3.00: configured for UDMA/33 ata4.00: configured for UDMA/33 scsi 2:0:0:0: CD-ROM VBOX CD-ROM 1.0 PQ: 0 ANSI: 5 scsi 3:0:0:0: CD-ROM VBOX CD-ROM 1.0 PQ: 0 ANSI: 5 sd 0:0:0:0: [sda] 33296384 512-byte logical blocks: (17.0 GB/15.8 GiB) sd 1:0:0:0: [sdb] 16777216 512-byte logical blocks: (8.58 GB/8.00 GiB) sd 1:0:0:0: [sdb] Write Protect is off sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sd 0:0:0:0: [sda] Write Protect is off sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sdb: sda: sda1 sda2 unknown partition table sd 1:0:0:0: [sdb] Attached SCSI disk sd 0:0:0:0: [sda] Attached SCSI disk sr0: scsi3-mmc drive: 32x/32x xa/form2 tray Uniform CD-ROM driver Revision: 3.20 sr1: scsi3-mmc drive: 32x/32x xa/form2 tray dracut: Scanning devices sda2 sdb for LVM logical volumes vg_mdt/lv_root vg_mdt/lv_swap dracut: inactive '/dev/lvm-MDS/p1' [7.90 GiB] inherit dracut: inactive '/dev/vg_mdt/lv_root' [12.54 GiB] inherit dracut: inactive '/dev/vg_mdt/lv_swap' [2.84 GiB] inherit EXT4-fs (dm-0): INFO: recovery required on readonly filesystem EXT4-fs (dm-0): write access will be enabled during recovery EXT4-fs (dm-0): recovery complete EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: dracut: Mounted root filesystem /dev/mapper/vg_mdt-lv_root SELinux: Disabled at runtime. type=1404 audit(1346991077.364:2): selinux=0 auid=4294967295 ses=4294967295 dracut: dracut: Switching root Welcome to CentOS Starting udev: udev: starting version 147 %Gsd 0:0:0:0: Attached scsi generic sg0 type 0 sd 1:0:0:0: Attached scsi generic sg1 type 0 sr 2:0:0:0: Attached scsi generic sg2 type 5 sr 3:0:0:0: Attached scsi generic sg3 type 5 e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI e1000: Copyright (c) 1999-2006 Intel Corporation. ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10 e1000 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 10 (level, low) -> IRQ 10 e1000 0000:00:03.0: eth0: (PCI:33MHz:32-bit) 08:00:27:e1:0d:9e e1000 0000:00:03.0: eth0: Intel(R) PRO/1000 Network Connection ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 9 e1000 0000:00:08.0: PCI INT A -> Link[LNKD] -> GSI 9 (level, low) -> IRQ 9 e1000 0000:00:08.0: eth1: (PCI:33MHz:32-bit) 08:00:27:3a:c4:e3 e1000 0000:00:08.0: eth1: Intel(R) PRO/1000 Network Connection Intel ICH 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 5 (level, low) -> IRQ 5 intel8x0_measure_ac97_clock: measured 50631 usecs (12704 samples) intel8x0: measured clock 250913 rejected intel8x0_measure_ac97_clock: measured 50939 usecs (12704 samples) intel8x0: measured clock 249396 rejected intel8x0_measure_ac97_clock: measured 50964 usecs (12704 samples) intel8x0: measured clock 249273 rejected intel8x0: clocking to 48000 piix4_smbus 0000:00:07.0: SMBus base address uninitialized - upgrade BIOS or use force_addr=0xaddr microcode: CPU0 sig=0x306a9, pf=0x1, revision=0x0 platform microcode: firmware: requesting intel-ucode/06-3a-09 Microcode Update Driver: v2.00 , Peter Oruba parport_pc 00:04: reported by Plug and Play ACPI ppdev: user-space parallel port driver [ OK ] Setting hostname mds2: [ OK ] Setting up Logical Volume Management: 1 logical volume(s) in volume group "lvm-MDS" now active 2 logical volume(s) in volume group "vg_mdt" now active [ OK ] Checking filesystems Checking all file systems. [/sbin/fsck.ext4 (1) -- /] fsck.ext4 -a /dev/mapper/vg_mdt-lv_root /dev/mapper/vg_mdt-lv_root: clean, 77487/822544 files, 2004901/3288064 blocks [/sbin/fsck.ext4 (1) -- /boot] fsck.ext4 -a /dev/sda1 /dev/sda1: recovering journal /dev/sda1: clean, 50/128016 files, 314383/512000 blocks [ OK ] Remounting root filesystem in read-write mode: [ OK ] Mounting local filesystems: EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: [ OK ] Enabling local filesystem quotas: [ OK ] Enabling /etc/fstab swaps: Adding 2981880k swap on /dev/mapper/vg_mdt-lv_swap. Priority:-1 extents:1 across:2981880k [ OK ] Entering non-interactive startup Calling the system activity data collector (sadc): Starting monitoring for VG lvm-MDS: 1 logical volume(s) in volume group "lvm-MDS" monitored [ OK ] Starting monitoring for VG vg_mdt: 2 logical volume(s) in volume group "vg_mdt" monitored [ OK ] NET: Registered protocol family 10 lo: Disabled Privacy Extensions Bringing up loopback interface: [ OK ] Bringing up interface eth1: e1000: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX ADDRCONF(NETDEV_UP): eth1: link is not ready ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready [ OK ] Starting auditd: [ OK ] Starting portrRPC: Registered named UNIX socket transport module. eserve: [ OK ] Starting system logger: [ OK ] Starting irqbalance: [ OK ] Starting rpcbind: [ OK ] Starting NFS statd: [ OK ] Starting RPC: Registered udp transport module. RPC idmapd: RPC: Registered tcp transport module. RPC: Registered tcp NFSv4.1 backchannel transport module. [ OK ] Starting kdump:[FAILED] Starting system message bus: [ OK ] Starting cups: [ OK ] Mounting other filesystems: [ OK ] Starting acpi daemon: [ OK ] Starting HAL daemon: [ OK ] Retrigger failed udev events[ OK ] Loading autofs4: [ OK ] Starting automount: [ OK ] Starting mcelog daemon [ OK ] Starting sshd: [ OK ] Starting postfix: [ OK ] Starting abrt daemon: [ OK ] Starting Qpid AMQP daemon: [ OK ] Starting crond: [ OK ] Starting atd: [ OK ] CentOS release 6.2 (Final) Kernel 2.6.32.masterDebugA on an x86_64 mds2 login: Stopping atd: [ OK ] Stopping cups: [ OK ] Stopping Qpid AMQP daemon: [ OK ] Stopping abrt daemon: [ OK ] Stopping sshd: [ OK ] Shutting down postfix: [ OK ] Stopping mcelog [ OK ] Stopping crond: [ OK ] Stopping automount: [ OK ] Stopping acpi daemon: [ OK ] Stopping HAL daemon: [ OK ] Stopping system message bus: [ OK ] Stopping rpcbind: [ OK ] Stopping auditd: type=1305 audit(1346993339.081:114): audit_pid=0 old=1004 auid=4294967295 ses=4294967295 res=1 [ OK ] type=1305 audit(1346993339.257:115): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1 Shutting down system logger: [ OK ] Shutting down interface eth1: [ OK ] Shutting down loopback interface: [ OK ] Stopping monitoring for VG lvm-MDS: 1 logical volume(s) in volume group "lvm-MDS" unmonitored [ OK ] Stopping monitoring for VG vg_mdt: 2 logical volume(s) in volume group "vg_mdt" unmonitored [ OK ] Sending all processes the TERM signal... [ OK ] Sending all processes the KILL signal... [ OK ] Saving random seed: [ OK ] Syncing hardware clock to system time [ OK ] Turning off swap: [ OK ] Turning off quotas: [ OK ] Unmounting pipe file systems: [ OK ] Unmounting file systems: [ OK ] init: Re-executing /sbin/init Please stand by while rebooting the system... md: stopping all md devices. sd 1:0:0:0: [sdb] Synchronizing SCSI cache sd 0:0:0:0: [sda] Synchronizing SCSI cache e1000 0000:00:08.0: PCI INT A disabled e1000 0000:00:03.0: PCI INT A disabled Restarting system. machine restart Initializing cgroup subsys cpuset Initializing cgroup subsys cpu Linux version 2.6.32.masterDEBUG11A (build@build63) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Thu Sep 6 20:21:34 PDT 2012 Command line: ro root=/dev/mapper/vg_mdt-lv_root nomodeset rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_mdt/lv_root rd_NO_MD crashkernel=auto rd_LVM_LV=vg_mdt/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM console=tty0 console=ttyS0,9800n8 SYSFONT=latarcyrheb-sun16 noapic KERNEL supported cpus: Intel GenuineIntel AMD AuthenticAMD Centaur CentaurHauls BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 000000000009fc00 (usable) BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved) BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 000000003fff0000 (usable) BIOS-e820: 000000003fff0000 - 0000000040000000 (ACPI data) BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved) DMI 2.5 present. SMBIOS version 2.5 @ 0xFFF30 last_pfn = 0x3fff0 max_arch_pfn = 0x400000000 x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 CPU MTRRs all blank - virtualized system. init_memory_mapping: 0000000000000000-000000003fff0000 RAMDISK: 37ac2000 - 37fefae4 ACPI: RSDP 00000000000e0000 00024 (v02 VBOX ) ACPI: XSDT 000000003fff0030 0003C (v01 VBOX VBOXXSDT 00000001 ASL 00000061) ACPI: FACP 000000003fff00f0 000F4 (v04 VBOX VBOXFACP 00000001 ASL 00000061) ACPI: DSDT 000000003fff0470 01B96 (v01 VBOX VBOXBIOS 00000002 INTL 20100528) ACPI: FACS 000000003fff0200 00040 ACPI: APIC 000000003fff0240 00054 (v02 VBOX VBOXAPIC 00000001 ASL 00000061) ACPI: SSDT 000000003fff02a0 001CC (v01 VBOX VBOXCPUT 00000002 INTL 20100528) No NUMA configuration found Faking a node at 0000000000000000-000000003fff0000 Bootmem setup node 0 0000000000000000-000000003fff0000 NODE_DATA [0000000000009000 - 000000000003cfff] bootmap [000000000003d000 - 0000000000044fff] pages 8 (7 early reservations) ==> bootmem [0000000000 - 003fff0000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000] #1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000] #2 [0001000000 - 0002012024] TEXT DATA BSS ==> [0001000000 - 0002012024] #3 [0037ac2000 - 0037fefae4] RAMDISK ==> [0037ac2000 - 0037fefae4] #4 [000009fc00 - 0000100000] BIOS reserved ==> [000009fc00 - 0000100000] #5 [0002013000 - 0002013079] BRK ==> [0002013000 - 0002013079] #6 [0000008000 - 0000009000] PGTABLE ==> [0000008000 - 0000009000] found SMP MP-table at [ffff88000009fff0] 9fff0 Zone PFN ranges: DMA 0x00000001 -> 0x00001000 DMA32 0x00001000 -> 0x00100000 Normal 0x00100000 -> 0x00100000 Movable zone start PFN for each node early_node_map[2] active PFN ranges 0: 0x00000001 -> 0x0000009f 0: 0x00000100 -> 0x0003fff0 ACPI: PM-Timer IO Port: 0x4008 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) ACPI: Skipping IOAPIC probe due to 'noapic' option. Using ACPI for processor (LAPIC) configuration information Intel MultiProcessor Specification v1.4 MPTABLE: OEM ID: VBOXCPU MPTABLE: Product ID: VirtualBox MPTABLE: APIC at: 0xFEE00000 I/O APIC #0 Version 17 at 0xFEC00000. Processors: 1 SMP: Allowing 1 CPUs, 0 hotplug CPUs PM: Registered nosave memory: 000000000009f000 - 00000000000a0000 PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000 PM: Registered nosave memory: 00000000000f0000 - 0000000000100000 Allocating PCI resources starting at 40000000 (gap: 40000000:bffc0000) Booting paravirtualized kernel on bare hardware NR_CPUS:4096 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1 PERCPU: Embedded 31 pages/cpu @ffff880002200000 s94488 r8192 d24296 u2097152 pcpu-alloc: s94488 r8192 d24296 u2097152 alloc=1*2097152 pcpu-alloc: [0] 0 Built 1 zonelists in Node order, mobility grouping on. Total pages: 258346 Policy zone: DMA32 Kernel command line: ro root=/dev/mapper/vg_mdt-lv_root nomodeset rd_NO_LUKS LANG=en_US.UTF-8 rd_LVM_LV=vg_mdt/lv_root rd_NO_MD rd_LVM_LV=vg_mdt/lv_swap KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM console=tty0 console=ttyS0,9800n8 SYSFONT=latarcyrheb-sun16 noapic PID hash table entries: 4096 (order: 3, 32768 bytes) Checking aperture... No AGP bridge found Memory: 1011392k/1048512k available (5160k kernel code, 392k absent, 36728k reserved, 7158k data, 1260k init) Hierarchical RCU implementation. NR_IRQS:33024 nr_irqs:256 Console: colour VGA+ 80x25 console [tty0] enabled console [ttyS0] enabled allocated 8388608 bytes of page_cgroup please try 'cgroup_disable=memory' option if you don't want memory cgroups Fast TSC calibration using PIT Detected 2285.062 MHz processor. Calibrating delay loop (skipped), value calculated using timer frequency.. 4570.12 BogoMIPS (lpj=2285062) pid_max: default: 32768 minimum: 301 Security Framework initialized SELinux: Initializing. Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes) Inode-cache hash table entries: 65536 (order: 7, 524288 bytes) Mount-cache hash table entries: 256 Initializing cgroup subsys ns Initializing cgroup subsys cpuacct Initializing cgroup subsys memory Initializing cgroup subsys devices Initializing cgroup subsys freezer Initializing cgroup subsys net_cls Initializing cgroup subsys blkio Initializing cgroup subsys perf_event Initializing cgroup subsys net_prio mce: CPU supports 0 MCE banks using mwait in idle threads. SMP alternatives: switching to UP code Freeing SMP alternatives: 34k freed ACPI: Core revision 20090903 ACPI: setting ELCR to 0200 (from 0e20) ftrace: converting mcount calls to 0f 1f 44 00 00 ftrace: allocating 21034 entries in 83 pages Setting APIC routing to flat CPU0: Intel(R) Core(TM) i7-3610QM CPU @ 2.30GHz stepping 09 Performance Events: unsupported p6 CPU model 58 no PMU driver, software events only. NMI watchdog disabled (cpu0): hardware events not enabled Brought up 1 CPUs Total of 1 processors activated (4570.12 BogoMIPS). devtmpfs: initialized regulator: core version 0.5 NET: Registered protocol family 16 ACPI: bus type pci registered PCI: Using configuration type 1 for base access bio: create slab at 0 ACPI: Executed 1 blocks of module-level executable AML code ACPI: Interpreter enabled ACPI: (supports S0 S5) ACPI: Using PIC for interrupt routing ACPI: No dock devices found. HEST: Table not found. PCI: Ignoring host bridge windows from ACPI; if necessary, use "pci=use_crs" and report a bug ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) ACPI: PCI Interrupt Link [LNKA] (IRQs *5 9 10 11) ACPI: PCI Interrupt Link [LNKB] (IRQs 5 9 10 *11) ACPI: PCI Interrupt Link [LNKC] (IRQs 5 9 *10 11) ACPI: PCI Interrupt Link [LNKD] (IRQs 5 *9 10 11) vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none vgaarb: loaded vgaarb: bridge control possible 0000:00:02.0 SCSI subsystem initialized usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb PCI: Using ACPI for IRQ routing NetLabel: Initializing NetLabel: domain hash size = 128 NetLabel: protocols = UNLABELED CIPSOv4 NetLabel: unlabeled traffic allowed by default Switching to clocksource jiffies pnp: PnP ACPI init ACPI: bus type pnp registered pnp: PnP ACPI: found 6 devices ACPI: ACPI bus type pnp unregistered Switching to clocksource acpi_pm NET: Registered protocol family 2 IP route cache hash table entries: 32768 (order: 6, 262144 bytes) TCP established hash table entries: 131072 (order: 9, 2097152 bytes) TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) TCP: Hash tables configured (established 131072 bind 65536) TCP reno registered NET: Registered protocol family 1 pci 0000:00:00.0: Limiting direct PCI/PCI transfers pci 0000:00:01.0: Activating ISA DMA hang workarounds ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11 pci 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, low) -> IRQ 11 pci 0000:00:06.0: PCI INT A disabled Trying to unpack rootfs image as initramfs... Freeing initrd memory: 5302k freed platform rtc_cmos: registered platform RTC device (no PNP device found) audit: initializing netlink socket (disabled) type=2000 audit(1346993353.461:1): initialized HugeTLB registered 2 MB page size, pre-allocated 0 pages VFS: Disk quotas dquot_6.5.2 Dquot-cache hash table entries: 512 (order 0, 4096 bytes) msgmni has been set to 1985 alg: No test for stdrng (krng) ksign: Installing public key data Loading keyring - Added public key 2CC9252176C603B4 - User ID: Red Hat, Inc. (Kernel Module GPG key) - Added public key D4A26C9CCD09BEDA - User ID: Red Hat Enterprise Linux Driver Update Program Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252) io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered (default) io scheduler cfq registered pci_hotplug: PCI Hot Plug PCI Core version: 0.5 pciehp: PCI Express Hot Plug Controller Driver version: 0.4 acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 ACPI: AC Adapter [AC] (on-line) input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 ACPI: Power Button [PWRF] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1 ACPI: Sleep Button [SLPF] ERST: Table is not found! GHES: HEST is not enabled! ACPI: Battery Slot [BAT0] (battery present) Non-volatile memory driver v1.3 Linux agpgart interface v0.103 crash memory driver: version 1.1 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled ˙serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A 00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A brd: module loaded loop: module loaded input: Macintosh mouse button emulation as /devices/virtual/input/input2 Fixed MDIO Bus: probed ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver ohci_hcd 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, low) -> IRQ 11 ohci_hcd 0000:00:06.0: OHCI Host Controller ohci_hcd 0000:00:06.0: new USB bus registered, assigned bus number 1 ohci_hcd 0000:00:06.0: irq 11, io mem 0xf0804000 usb usb1: New USB device found, idVendor=1d6b, idProduct=0001 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: OHCI Host Controller usb usb1: Manufacturer: Linux 2.6.32.masterDEBUG11A ohci_hcd usb usb1: SerialNumber: 0000:00:06.0 usb usb1: configuration #1 chosen from 1 choice hub 1-0:1.0: USB hub found hub 1-0:1.0: 8 ports detected uhci_hcd: USB Universal Host Controller Interface driver PNP: PS/2 Controller [PNP0303:PS2K,PNP0f03:PS2M] at 0x60,0x64 irq 1,12 serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX port at 0x60,0x64 irq 12 mice: PS/2 mouse device common for all mice input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3 rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 rtc0: alarms up to one day, 114 bytes nvram cpuidle: using governor ladder cpuidle: using governor menu EFI Variables Facility v0.08 2004-May-17 input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input4 usbcore: registered new interface driver hiddev usbcore: registered new interface driver usbhid usbhid: v2.6:USB HID core driver TCP cubic registered Initializing XFRM netlink socket NET: Registered protocol family 17 registered taskstats version 1 rtc_cmos rtc_cmos: setting system clock to 2012-09-07 04:49:16 UTC (1346993356) Initalizing network drop monitor service Freeing unused kernel memory: 1260k freed Write protecting the kernel read-only data: 10240k Freeing unused kernel memory: 964k freed Freeing unused kernel memory: 1728k freed dracut: dracut-004-256.el6 dracut: rd_NO_LUKS: removing cryptoluks activation device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.22.6-ioctl (2011-10-19) initialised: dm-devel@redhat.com udev: starting version 147 dracut: Starting plymouth daemon %GRefined TSC clocksource calibration: 2285.611 MHz. Switching to clocksource tsc %Gscsi0 : ata_piix scsi1 : ata_piix ata1: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6 bmdma 0xd000 irq 14 ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0xd008 irq 15 ata1.00: ATAPI: VBOX CD-ROM, 1.0, max UDMA/133 ata2.00: ATAPI: VBOX CD-ROM, 1.0, max UDMA/133 ata2.00: configured for UDMA/33 ata1.00: configured for UDMA/33 scsi 0:0:0:0: CD-ROM VBOX CD-ROM 1.0 PQ: 0 ANSI: 5 scsi 1:0:0:0: CD-ROM VBOX CD-ROM 1.0 PQ: 0 ANSI: 5 ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 5 ahci 0000:00:0d.0: PCI INT A -> Link[LNKA] -> GSI 5 (level, low) -> IRQ 5 ahci: SSS flag set, parallel bus scan disabled ahci 0000:00:0d.0: AHCI 0001.0100 32 slots 2 ports 3 Gbps 0x3 impl SATA mode ahci 0000:00:0d.0: flags: 64bit ncq stag only ccc scsi2 : ahci scsi3 : ahci ata3: SATA max UDMA/133 abar m8192@0xf0840000 port 0xf0840100 irq 5 ata4: SATA max UDMA/133 abar m8192@0xf0840000 port 0xf0840180 irq 5 ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata3.00: ATA-6: VBOX HARDDISK, 1.0, max UDMA/133 ata3.00: 33296384 sectors, multi 128: LBA48 NCQ (depth 31/32) ata3.00: configured for UDMA/133 scsi 2:0:0:0: Direct-Access ATA VBOX HARDDISK 1.0 PQ: 0 ANSI: 5 ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300) ata4.00: ATA-6: VBOX HARDDISK, 1.0, max UDMA/133 ata4.00: 16777216 sectors, multi 128: LBA48 NCQ (depth 31/32) ata4.00: configured for UDMA/133 scsi 3:0:0:0: Direct-Access ATA VBOX HARDDISK 1.0 PQ: 0 ANSI: 5 sr0: scsi3-mmc drive: 32x/32x xa/form2 tray Uniform CD-ROM driver Revision: 3.20 sr1: scsi3-mmc drive: 32x/32x xa/form2 tray sd 2:0:0:0: [sda] 33296384 512-byte logical blocks: (17.0 GB/15.8 GiB) sd 3:0:0:0: [sdb] 16777216 512-byte logical blocks: (8.58 GB/8.00 GiB) sd 3:0:0:0: [sdb] Write Protect is off sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sd 2:0:0:0: [sda] Write Protect is off sdb: sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sda: sda1 sda2 unknown partition table sd 3:0:0:0: [sdb] Attached SCSI disk sd 2:0:0:0: [sda] Attached SCSI disk dracut: Scanning devices sda2 sdb for LVM logical volumes vg_mdt/lv_root vg_mdt/lv_swap dracut: inactive '/dev/lvm-MDS/p1' [7.90 GiB] inherit dracut: inactive '/dev/vg_mdt/lv_root' [12.54 GiB] inherit dracut: inactive '/dev/vg_mdt/lv_swap' [2.84 GiB] inherit EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: dracut: Mounted root filesystem /dev/mapper/vg_mdt-lv_root SELinux: Disabled at runtime. type=1404 audit(1346993360.889:2): selinux=0 auid=4294967295 ses=4294967295 dracut: dracut: Switching root Welcome to CentOS readahead-collector: starting Starting udev: udev: starting version 147 %Gsr 0:0:0:0: Attached scsi generic sg0 type 5 sr 1:0:0:0: Attached scsi generic sg1 type 5 sd 2:0:0:0: Attached scsi generic sg2 type 0 sd 3:0:0:0: Attached scsi generic sg3 type 0 e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI e1000: Copyright (c) 1999-2006 Intel Corporation. ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10 e1000 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 10 (level, low) -> IRQ 10 e1000 0000:00:03.0: eth0: (PCI:33MHz:32-bit) 08:00:27:e1:0d:9e e1000 0000:00:03.0: eth0: Intel(R) PRO/1000 Network Connection ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 9 e1000 0000:00:08.0: PCI INT A -> Link[LNKD] -> GSI 9 (level, low) -> IRQ 9 e1000 0000:00:08.0: eth1: (PCI:33MHz:32-bit) 08:00:27:3a:c4:e3 e1000 0000:00:08.0: eth1: Intel(R) PRO/1000 Network Connection Intel ICH 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 5 (level, low) -> IRQ 5 intel8x0_measure_ac97_clock: measured 50893 usecs (11786 samples) intel8x0: measured clock 231583 rejected intel8x0_measure_ac97_clock: measured 51073 usecs (11334 samples) intel8x0: measured clock 221917 rejected intel8x0_measure_ac97_clock: measured 50965 usecs (11334 samples) intel8x0: measured clock 222387 rejected intel8x0: clocking to 48000 piix4_smbus 0000:00:07.0: SMBus base address uninitialized - upgrade BIOS or use force_addr=0xaddr microcode: CPU0 sig=0x306a9, pf=0x1, revision=0x0 platform microcode: firmware: requesting intel-ucode/06-3a-09 Microcode Update Driver: v2.00 , Peter Oruba parport_pc 00:04: reported by Plug and Play ACPI ppdev: user-space parallel port driver [ OK ] Setting hostname mds2: [ OK ] Setting up Logical Volume Management: 1 logical volume(s) in volume group "lvm-MDS" now active 2 logical volume(s) in volume group "vg_mdt" now active [ OK ] Checking filesystems Checking all file systems. [/sbin/fsck.ext4 (1) -- /] fsck.ext4 -a /dev/mapper/vg_mdt-lv_root /dev/mapper/vg_mdt-lv_root: clean, 79722/822544 files, 2395800/3288064 blocks [/sbin/fsck.ext4 (1) -- /boot] fsck.ext4 -a /dev/sda1 /dev/sda1: clean, 55/128016 files, 369757/512000 blocks [ OK ] Remounting root filesystem in read-write mode: [ OK ] Mounting local filesystems: EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: [ OK ] Enabling local filesystem quotas: [ OK ] Enabling /etc/fstab swaps: Adding 2981880k swap on /dev/mapper/vg_mdt-lv_swap. Priority:-1 extents:1 across:2981880k [ OK ] Entering non-interactive startup readahead-disable-service: delaying service auditd Calling the system activity data collector (sadc): Starting monitoring for VG lvm-MDS: 1 logical volume(s) in volume group "lvm-MDS" monitored [ OK ] Starting monitoring for VG vg_mdt: 2 logical volume(s) in volume group "vg_mdt" monitored [ OK ] NET: Registered protocol family 10 lo: Disabled Privacy Extensions Bringing up loopback interface: [ OK ] Bringing up interface eth1: e1000: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX ADDRCONF(NETDEV_UP): eth1: link is not ready ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready [ OK ] Starting portreserve: [ OK ] Starting system logger: [ OK ] Starting irqbalance: [ OK ] Starting rpcbind: [ OK ] Starting NFS statd: [ OK ] Starting RPC idmapd: RPC: Registered named UNIX socket transport module. RPC: Registered udp transport module. RPC: Registered tcp transport module. RPC: Registered tcp NFSv4.1 backchannel transport module. [ OK ] Starting kdump:[FAILED] Starting system message bus: [ OK ] Starting cups: [ OK ] Mounting other filesystems: [ OK ] Starting acpi daemon: [ OK ] Starting HAL daemon: [ OK ] Retrigger failed udev events[ OK ] Loading autofs4: [ OK ] Starting automount: [ OK ] Starting mcelog daemon [ OK ] Starting sshd: [ OK ] Starting postfix: [ OK ] Starting abrt daemon: [ OK ] Starting Qpid AMQP daemon: [ OK ] Starting crond: [ OK ] Starting atd: [ OK ] LNet: HW CPU cores: 1, npartitions: 1 alg: No test for crc32 (crc32-table) alg: No test for adler32 (adler32-zlib) CentOS release 6.2 (Final) Kernel 2.6.32.masterDEBUG11A on an x86_64 mds2 login: padlock: VIA PadLock Hash Engine not detected. LNet: Added LNI 192.168.56.104@tcp [8/256/0/180] LNet: Accept secure, port 988 Lustre: Lustre: Build Version: 2.3.50-g0db6ebb-CHANGED-2.6.32.masterDEBUG11A type=1305 audit(1346993403.808:17274): auid=4294967295 ses=4294967295 op="remove rule" key=(null) list=4 res=1 type=1305 audit(1346993403.808:17275): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1 readahead-collector: starting delayed service auditd readahead-collector: sorting readahead-collector: finished LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: EXT4-fs (dm-2): Couldn't mount because of unsupported optional features (1000) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Setting parameter lustre-clilov.lov.stripesize in log lustre-client Lustre: lustre-MDT0000: new disk, initializing Lustre: 2638:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 2668:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0001_UUID now active, resetting orphans Lustre: 2668:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0004_UUID now active, resetting orphans Lustre: Skipped 2 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:04:28 (1346994268) Lustre: 2856:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 22:05:41 (1346994341) Lustre: MGS has stopped. LustreError: 2942:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 2942:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 2942:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 2942:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 Lustre: 2942:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346994349/real 1346994349] req@ffff88002f2600d8 x1412424954552005/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346994355 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 8 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: 3127:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 3153:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 2 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:09:01 (1346994541) Lustre: 3334:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 22:10:12 (1346994612) Lustre: MGS has stopped. LustreError: 3416:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3416:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 Lustre: 3416:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346994619/real 1346994619] req@ffff88003cf820d8 x1412424954559868/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346994625 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages hrtimer: interrupt took 3175978 ns Lustre: lustre-MDT0000: new disk, initializing Lustre: 3600:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 3626:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: 3626:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0006_UUID now active, resetting orphans Lustre: Skipped 5 previous similar messages Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:12:52 (1346994772) Lustre: 3807:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 22:14:03 (1346994843) Lustre: MGS has stopped. LustreError: 3886:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3886:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 3886:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 3886:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 3886:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346994850/real 1346994850] req@ffff880024d7cac8 x1412424954567733/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346994856 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: 4066:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 4092:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:15:49 (1346994949) Lustre: 4273:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 22:17:01 (1346995021) Lustre: MGS has stopped. LustreError: 4352:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 4352:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 4352:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 4352:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 4352:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346995028/real 1346995028] req@ffff880024c54d20 x1412424954575605/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346995034 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: 4533:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 4559:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 6 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:18:49 (1346995129) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 22:20:00 (1346995200) Lustre: MGS has stopped. LustreError: 4821:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 4821:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 4821:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 4821:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 4821:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346995207/real 1346995207] req@ffff88002ee96e20 x1412424954583484/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346995213 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: 5001:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5001:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 1 previous similar message Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 5027:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:21:42 (1346995302) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 22:22:53 (1346995373) Lustre: MGS has stopped. LustreError: 5286:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 5286:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 5286:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 5286:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 5286:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346995381/real 1346995381] req@ffff880031bd3278 x1412424954591351/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346995387 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: 5466:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5466:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 1 previous similar message Lustre: 5492:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:24:35 (1346995475) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 22:25:47 (1346995547) Lustre: MGS has stopped. LustreError: 5751:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 5751:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 5751:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 5751:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 5751:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346995555/real 1346995555] req@ffff880024e82ca0 x1412424954599213/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346995561 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: 5958:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 5958:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 1 previous similar message Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:27:28 (1346995648) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 22:28:39 (1346995719) Lustre: MGS has stopped. LustreError: 6218:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6218:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 6218:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 6218:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 6218:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346995727/real 1346995727] req@ffff88002a0f3178 x1412424954607085/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346995733 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: 6398:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 6398:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 3 previous similar messages Lustre: 6424:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 6424:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 1 previous similar message LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:30:26 (1346995826) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 22:31:37 (1346995897) Lustre: MGS has stopped. LustreError: 6685:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6685:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 6685:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 6685:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 6685:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346995905/real 1346995905] req@ffff88002a90f238 x1412424954614949/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346995911 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 9 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 13 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:33:19 (1346995999) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 22:34:31 (1346996071) Lustre: MGS has stopped. LustreError: 7150:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 7150:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 1 previous similar message LustreError: 7150:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 7150:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 1 previous similar message Lustre: 7150:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346996075/real 1346996075] req@ffff880031bd2f60 x1412424954622812/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346996081 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: lustre-MDT0000: new disk, initializing Lustre: 7356:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 7356:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:36:10 (1346996170) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 22:37:20 (1346996240) Lustre: MGS has stopped. Lustre: 7616:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346996247/real 1346996247] req@ffff88002a8d7338 x1412424954630665/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346996253 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: server umount lustre-MDT0000 complete LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 19 previous similar messages Lustre: 7797:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 7797:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 5 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:39:05 (1346996345) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 22:40:17 (1346996417) LustreError: 8083:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 8083:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 3 previous similar messages LustreError: 8083:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 8083:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 1 previous similar message Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 1 previous similar message Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 20 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 1 previous similar message Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:41:57 (1346996517) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 22:43:08 (1346996588) Lustre: MGS has stopped. Lustre: Skipped 1 previous similar message Lustre: 8548:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346996596/real 1346996596] req@ffff880024e990b8 x1412424954646379/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346996602 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 8548:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 1 previous similar message Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 1 previous similar message LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 8754:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 8754:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:44:51 (1346996691) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 22:46:02 (1346996762) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 29 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:47:50 (1346996870) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 22:49:01 (1346996941) LustreError: 9482:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 9482:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 5 previous similar messages LustreError: 9482:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 9482:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 5 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 2 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 2 previous similar messages Lustre: 9665:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 9665:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 2 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:50:48 (1346997048) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 22:51:59 (1346997119) Lustre: MGS has stopped. Lustre: Skipped 2 previous similar messages Lustre: 9950:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346997127/real 1346997127] req@ffff880024c2ace0 x1412424954669984/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346997133 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 9950:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 2 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 2 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:53:43 (1346997223) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 22:54:54 (1346997294) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 10623:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 10623:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:56:37 (1346997397) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 22:57:49 (1346997469) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:59:33 (1346997573) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 23:00:44 (1346997644) LustreError: 11349:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 11349:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 11349:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 11349:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 11548:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 11548:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:02:34 (1346997754) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 23:03:45 (1346997825) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 11835:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346997832/real 1346997832] req@ffff8800251670b8 x1412424954701469/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346997838 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 11835:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:05:28 (1346997928) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 23:06:39 (1346997999) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 12506:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 12506:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:08:25 (1346998105) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 23:09:38 (1346998178) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:11:16 (1346998276) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 23:12:28 (1346998348) LustreError: 13234:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 13234:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 13234:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 13234:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 13414:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 13414:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:14:12 (1346998452) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 23:15:23 (1346998523) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 13699:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346998531/real 1346998531] req@ffff880025646da0 x1412424954732925/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346998537 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 13699:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:17:09 (1346998629) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 23:18:20 (1346998700) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 14371:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 14371:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:20:02 (1346998802) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 23:21:15 (1346998875) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:22:56 (1346998976) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 23:24:08 (1346999048) LustreError: 15098:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 15098:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 15098:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 15098:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 15278:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 15278:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:25:54 (1346999154) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 23:27:07 (1346999227) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 15564:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346999231/real 1346999231] req@ffff880024c7f138 x1412424954764394/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346999237 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 15564:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:28:43 (1346999323) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 23:29:55 (1346999395) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 16239:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 16239:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:31:39 (1346999499) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 23:32:52 (1346999572) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:34:38 (1346999678) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 23:35:49 (1346999749) LustreError: 16965:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 16965:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 16965:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 16965:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 17145:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 17145:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:37:34 (1346999854) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 23:38:45 (1346999925) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 17431:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1346999932/real 1346999932] req@ffff88003d95c158 x1412424954795835/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1346999938 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 17431:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:40:34 (1347000034) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 23:41:45 (1347000105) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 18104:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 18104:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:43:26 (1347000206) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 23:44:37 (1347000277) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:46:23 (1347000383) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 23:47:34 (1347000454) LustreError: 18831:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 18831:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 18831:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 18831:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 19011:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 19011:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:49:17 (1347000557) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 23:50:28 (1347000628) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 19297:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347000636/real 1347000636] req@ffff8800297ff178 x1412424954827315/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347000642 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 19297:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:52:12 (1347000732) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 23:53:23 (1347000803) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 19976:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 19976:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:55:09 (1347000909) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 23:56:20 (1347000980) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:58:02 (1347001082) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 23:59:13 (1347001153) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LustreError: 20701:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 20701:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 20701:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 20701:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 20883:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 20883:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:01:09 (1347001269) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 00:02:20 (1347001340) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 21192:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347001356/real 1347001356] req@ffff88002a750158 x1412424954858830/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347001362 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 21192:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:04:10 (1347001450) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 00:05:22 (1347001522) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 21864:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 21864:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:07:05 (1347001625) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 00:08:17 (1347001697) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:09:59 (1347001799) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 00:11:12 (1347001872) LustreError: 22592:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 22592:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 22592:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 22592:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 22772:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 22772:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:12:56 (1347001976) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 00:14:07 (1347002047) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 23057:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347002055/real 1347002055] req@ffff880028396ca0 x1412424954890309/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347002061 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 23057:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:15:53 (1347002153) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 00:17:04 (1347002224) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 23729:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 23729:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:18:48 (1347002328) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 00:20:00 (1347002400) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:21:44 (1347002504) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 00:22:56 (1347002576) LustreError: 24456:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 24456:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 24456:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 24456:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 24636:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 24636:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:24:44 (1347002684) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 00:25:55 (1347002755) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 24922:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347002762/real 1347002762] req@ffff8800253cd0b8 x1412424954921795/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347002768 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 24922:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:27:38 (1347002858) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 00:28:50 (1347002930) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 25596:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 25596:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:30:31 (1347003031) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 00:31:42 (1347003102) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:33:29 (1347003209) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 00:34:40 (1347003280) LustreError: 26321:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 26321:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 26321:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 26321:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 26501:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 26501:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:36:20 (1347003380) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 00:37:32 (1347003452) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 26788:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347003460/real 1347003460] req@ffff88002a8d7178 x1412424954953278/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347003466 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 26788:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:39:17 (1347003557) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 00:40:29 (1347003629) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 27460:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 27460:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:42:16 (1347003736) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 00:43:28 (1347003808) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:45:10 (1347003910) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 00:46:22 (1347003982) LustreError: 28187:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 28187:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 28187:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 28187:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 28367:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 28367:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:48:12 (1347004092) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 00:49:22 (1347004162) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 28652:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347004170/real 1347004170] req@ffff880024d70e60 x1412424954984770/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347004176 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 28652:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:51:07 (1347004267) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 00:52:18 (1347004338) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 29326:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 29326:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:54:00 (1347004440) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 00:55:13 (1347004513) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:56:54 (1347004614) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 00:58:06 (1347004686) LustreError: 30051:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 30051:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 30051:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 30051:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 30231:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 30231:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:59:53 (1347004793) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 01:01:04 (1347004864) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 30542:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347004872/real 1347004872] req@ffff88003ca76118 x1412424955016226/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347004878 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 30542:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:02:49 (1347004969) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 01:04:01 (1347005041) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 31214:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 31214:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:05:49 (1347005149) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 01:07:00 (1347005220) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:08:41 (1347005321) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 01:09:52 (1347005392) LustreError: 31940:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 31940:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 31940:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 31940:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 32123:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 32123:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:11:37 (1347005497) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 01:12:49 (1347005569) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 32408:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347005577/real 1347005577] req@ffff88002ab551b8 x1412424955047712/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347005583 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 32408:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:14:32 (1347005672) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 01:15:43 (1347005743) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 616:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 616:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:17:36 (1347005856) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 01:18:47 (1347005927) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:20:33 (1347006033) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 01:21:45 (1347006105) LustreError: 1380:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 1380:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 1380:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 1380:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 1587:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 1587:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:23:28 (1347006208) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 01:24:40 (1347006280) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 1873:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347006289/real 1347006289] req@ffff88002572cde0 x1412424955079195/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347006295 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 1873:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:26:23 (1347006383) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 01:27:35 (1347006455) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 2547:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 2547:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:29:26 (1347006566) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 01:30:38 (1347006638) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:32:19 (1347006739) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 01:33:30 (1347006810) LustreError: 3274:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3274:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 3274:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 3274:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 3454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 3454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:35:16 (1347006916) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 01:36:27 (1347006987) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 3740:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347006995/real 1347006995] req@ffff880024c54de0 x1412424955110662/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347007001 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 3740:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:38:14 (1347007094) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 01:39:26 (1347007166) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 4414:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 4414:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 3 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:41:11 (1347007271) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 01:42:21 (1347007341) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:44:01 (1347007441) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 01:45:13 (1347007513) LustreError: 5141:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 5141:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 5141:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 5141:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 5321:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5321:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:46:56 (1347007616) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 01:48:09 (1347007689) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 5606:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347007693/real 1347007693] req@ffff880026a09510 x1412424955142126/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347007699 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 5606:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:49:58 (1347007798) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 01:51:09 (1347007869) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 6281:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 6281:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:52:55 (1347007975) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 01:54:07 (1347008047) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:55:53 (1347008153) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 01:57:05 (1347008225) LustreError: 7007:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 7007:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 7007:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 7007:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 7187:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 7187:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:58:52 (1347008332) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 02:00:05 (1347008405) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 7475:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347008409/real 1347008409] req@ffff880026cfd138 x1412424955173589/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347008415 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 7475:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:01:48 (1347008508) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 02:03:00 (1347008580) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 8169:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 8169:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:04:49 (1347008689) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 02:06:01 (1347008761) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:07:48 (1347008868) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:08:59 (1347008939) LustreError: 8895:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 8895:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 8895:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 8895:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 9077:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 9077:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:10:43 (1347009043) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 02:11:55 (1347009115) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 9362:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347009119/real 1347009119] req@ffff88002aa6f550 x1412424955205082/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347009125 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 9362:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:13:36 (1347009216) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 02:14:45 (1347009285) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 10035:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 10035:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:16:31 (1347009391) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 02:17:43 (1347009463) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:19:31 (1347009571) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 02:20:41 (1347009641) LustreError: 10761:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 10761:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 10761:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 10761:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 10941:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 10941:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:22:27 (1347009747) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:23:38 (1347009818) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 11229:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347009826/real 1347009826] req@ffff88003c9552b8 x1412424955236557/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347009832 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 11229:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:25:25 (1347009925) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 02:26:38 (1347009998) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 11900:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 11900:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:28:21 (1347010101) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 02:29:33 (1347010173) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:31:19 (1347010279) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:32:31 (1347010351) LustreError: 12628:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 12628:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 12628:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 12628:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 12808:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 12808:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:34:14 (1347010454) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:35:27 (1347010527) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 13093:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347010531/real 1347010531] req@ffff880025ca4da0 x1412424955268034/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347010537 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 13093:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:37:02 (1347010622) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 02:38:14 (1347010694) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 13766:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 13766:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:40:01 (1347010801) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 02:41:12 (1347010872) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:42:54 (1347010974) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 02:44:06 (1347011046) LustreError: 14494:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 14494:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 14494:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 14494:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 14674:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 14674:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:45:48 (1347011148) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:47:00 (1347011220) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 14959:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347011228/real 1347011228] req@ffff88003ca69378 x1412424955299512/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347011234 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 14959:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:48:43 (1347011323) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:49:55 (1347011395) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 15636:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 15636:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:51:40 (1347011500) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:52:51 (1347011571) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:54:32 (1347011672) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 02:55:45 (1347011745) LustreError: 16360:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 16360:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 16360:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 16360:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 16540:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 16540:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:57:31 (1347011851) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 02:58:42 (1347011922) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 16828:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347011930/real 1347011930] req@ffff880025f2a948 x1412424955330997/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347011936 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 16828:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:00:30 (1347012030) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 03:01:42 (1347012102) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 17523:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 17523:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:03:23 (1347012203) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 03:04:34 (1347012274) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:06:17 (1347012377) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 03:07:28 (1347012448) LustreError: 18251:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 18251:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 18251:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 18251:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 18431:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 18431:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:09:15 (1347012555) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 03:10:26 (1347012626) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 18718:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347012635/real 1347012635] req@ffff88002a93eca0 x1412424955362463/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347012641 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 18718:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:12:14 (1347012734) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 03:13:26 (1347012806) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 19390:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 19390:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:15:05 (1347012905) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 03:16:18 (1347012978) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:17:55 (1347013075) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 03:19:07 (1347013147) LustreError: 20115:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 20115:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 20115:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 20115:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 20297:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 20297:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:20:50 (1347013250) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 03:22:02 (1347013322) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 20583:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347013330/real 1347013330] req@ffff88003dd489c8 x1412424955393920/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347013336 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 20583:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:23:48 (1347013428) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 03:25:00 (1347013500) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 21256:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 21256:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:26:58 (1347013618) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 03:28:10 (1347013690) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:29:56 (1347013796) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 03:31:08 (1347013868) LustreError: 23641:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 23641:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 23641:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 23641:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 23821:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 23821:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:32:53 (1347013973) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 03:34:04 (1347014044) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 24106:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347014052/real 1347014052] req@ffff8800279df1b8 x1412424955425408/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347014058 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 24106:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:35:48 (1347014148) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 03:36:59 (1347014219) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 24778:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 24778:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:38:43 (1347014323) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 03:39:55 (1347014395) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:41:36 (1347014496) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 03:42:46 (1347014566) LustreError: 25507:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 25507:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 25507:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 25507:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 25687:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 25687:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:44:35 (1347014675) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 03:45:46 (1347014746) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 25973:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347014755/real 1347014755] req@ffff880024ca6ee0 x1412424955456889/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347014761 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 25973:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:47:32 (1347014852) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 03:48:44 (1347014924) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 26645:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 26645:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:50:26 (1347015026) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 03:51:38 (1347015098) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:53:19 (1347015199) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 03:54:31 (1347015271) LustreError: 27372:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 27372:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 27372:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 27372:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 27553:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 27553:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:56:19 (1347015379) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 03:57:29 (1347015449) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 27839:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347015458/real 1347015458] req@ffff880025660d60 x1412424955488384/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347015464 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 27839:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:59:17 (1347015557) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 04:00:30 (1347015630) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 28529:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 28529:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:02:15 (1347015735) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 04:03:27 (1347015807) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:05:09 (1347015909) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 04:06:20 (1347015980) LustreError: 29255:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 29255:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 29255:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 29255:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 29435:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 29435:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:08:03 (1347016083) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 04:09:15 (1347016155) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 29720:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347016164/real 1347016164] req@ffff8800294f51b8 x1412424955519851/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347016170 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 29720:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:10:58 (1347016258) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 04:12:10 (1347016330) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 30394:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 30394:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:13:54 (1347016434) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 04:15:06 (1347016506) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:16:46 (1347016606) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 04:17:57 (1347016677) LustreError: 31119:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 31119:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 31119:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 31119:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 31299:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 31299:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:19:41 (1347016781) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 04:20:53 (1347016853) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 31586:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347016862/real 1347016862] req@ffff88002a093238 x1412424955551318/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347016868 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 31586:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:22:39 (1347016959) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 04:23:51 (1347017031) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 32258:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 32258:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:25:37 (1347017137) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 04:26:49 (1347017209) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:28:32 (1347017312) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 04:29:45 (1347017385) LustreError: 519:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 519:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 519:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 519:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 705:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 705:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:31:24 (1347017484) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 04:32:35 (1347017555) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 999:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347017564/real 1347017564] req@ffff880031bdaf60 x1412424955582760/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347017570 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 999:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:34:18 (1347017658) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 04:35:31 (1347017731) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 1722:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 1722:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:37:13 (1347017833) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 04:38:24 (1347017904) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:40:10 (1347018010) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 04:41:22 (1347018082) LustreError: 2452:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 2452:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 2452:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 2452:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 2633:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 2633:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:43:10 (1347018190) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 04:44:22 (1347018262) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 2918:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347018270/real 1347018270] req@ffff880025f7b338 x1412424955614231/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347018276 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 2918:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:46:05 (1347018365) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 04:47:16 (1347018436) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 3590:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 3590:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:48:56 (1347018536) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 04:50:05 (1347018605) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:51:47 (1347018707) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 04:52:57 (1347018777) LustreError: 4318:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 4318:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 4318:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 4318:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 4498:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 4498:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:54:44 (1347018884) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 04:55:55 (1347018955) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 4784:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347018964/real 1347018964] req@ffff880026731178 x1412424955645668/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347018970 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 4784:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:57:40 (1347019060) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 04:58:51 (1347019131) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 5458:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 5458:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:00:38 (1347019238) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 05:01:51 (1347019311) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:03:43 (1347019423) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 120 sec == 05:04:56 (1347019496) LustreError: 6202:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6202:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 6202:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 6202:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 6382:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 6382:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:06:34 (1347019594) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 05:07:44 (1347019664) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 6667:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347019673/real 1347019673] req@ffff88002f04aca0 x1412424955677116/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347019679 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 6667:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:09:27 (1347019767) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 05:10:39 (1347019839) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 7339:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 7339:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:12:20 (1347019940) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 05:13:32 (1347020012) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:15:14 (1347020114) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 05:16:26 (1347020186) LustreError: 8066:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 8066:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 8066:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 8066:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 8246:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 8246:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:18:08 (1347020288) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 05:19:19 (1347020359) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 8531:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347020368/real 1347020368] req@ffff880025998b48 x1412424955708593/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347020374 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 8531:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:21:01 (1347020461) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 05:22:13 (1347020533) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: Skipped 1 previous similar message Lustre: 9205:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 9205:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:24:03 (1347020643) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 05:25:14 (1347020714) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:26:56 (1347020816) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 05:28:07 (1347020887) LustreError: 9930:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 9930:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 9930:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 9930:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 10110:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 10110:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:29:48 (1347020988) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 05:30:59 (1347021059) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 10399:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347021068/real 1347021068] req@ffff8800296248c8 x1412424955740075/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347021074 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 10399:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:32:49 (1347021169) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 05:34:01 (1347021241) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 11072:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 11072:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:35:43 (1347021343) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 05:36:56 (1347021416) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:38:46 (1347021526) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 120 sec == 05:39:59 (1347021599) LustreError: 11801:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 11801:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 11801:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 11801:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 11981:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 11981:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:41:40 (1347021700) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 05:42:51 (1347021771) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 12266:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347021780/real 1347021780] req@ffff8800314e7278 x1412424955771569/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347021786 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 12266:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:44:33 (1347021873) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 05:45:44 (1347021944) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: Skipped 1 previous similar message Lustre: 12938:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 12938:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:47:29 (1347022049) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 05:48:40 (1347022120) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:50:21 (1347022221) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 05:51:32 (1347022292) LustreError: 13666:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 13666:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 13666:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 13666:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 13847:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 13847:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:53:21 (1347022401) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 05:54:33 (1347022473) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 14133:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347022490/real 1347022490] req@ffff880031bb8ce0 x1412424955803050/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347022496 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 14133:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:56:25 (1347022585) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 05:57:37 (1347022657) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 14805:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 14805:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:59:18 (1347022758) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 06:00:30 (1347022830) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:02:23 (1347022943) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 06:03:34 (1347023014) LustreError: 15549:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 15549:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 15549:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 15549:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 15730:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 15730:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:05:18 (1347023118) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 06:06:29 (1347023189) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 16016:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347023198/real 1347023198] req@ffff88002725eee0 x1412424955834519/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347023204 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 16016:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:08:13 (1347023293) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 06:09:26 (1347023366) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: Skipped 1 previous similar message Lustre: 16690:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 16690:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:11:16 (1347023476) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 06:12:28 (1347023548) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:14:11 (1347023651) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 06:15:22 (1347023722) LustreError: 17417:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 17417:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 17417:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 17417:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 17597:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 17597:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:17:07 (1347023827) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 06:18:19 (1347023899) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 17883:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347023908/real 1347023908] req@ffff8800275a7238 x1412424955866022/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347023914 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 17883:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:20:04 (1347024004) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 06:21:15 (1347024075) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 18558:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 18558:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:22:59 (1347024179) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 06:24:11 (1347024251) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:26:00 (1347024360) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 06:27:10 (1347024430) LustreError: 19283:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 19283:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 19283:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 19283:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 19463:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 19463:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:28:53 (1347024533) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 06:30:04 (1347024604) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 19751:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347024613/real 1347024613] req@ffff880024d43710 x1412424955897500/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347024619 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 19751:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:31:55 (1347024715) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 121 sec == 06:33:08 (1347024788) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 20422:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 20422:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:35:00 (1347024900) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 121 sec == 06:36:12 (1347024972) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:38:00 (1347025080) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 06:39:12 (1347025152) LustreError: 21148:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 21148:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 21148:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 21148:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 21331:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 21331:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:41:00 (1347025260) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 06:42:12 (1347025332) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 21617:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347025341/real 1347025341] req@ffff880025f52f60 x1412424955929001/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347025347 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 21617:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:43:55 (1347025435) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 06:45:07 (1347025507) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 22290:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 22290:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:46:54 (1347025614) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 06:48:05 (1347025685) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:49:49 (1347025789) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 06:51:00 (1347025860) LustreError: 23016:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 23016:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 23016:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 23016:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 23196:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 23196:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:52:47 (1347025967) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 06:53:58 (1347026038) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 23482:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347026047/real 1347026047] req@ffff8800272d0f60 x1412424955960488/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347026053 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 23482:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:55:41 (1347026141) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 06:56:53 (1347026213) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 24153:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 24153:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:58:36 (1347026316) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 06:59:47 (1347026387) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:01:34 (1347026494) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 07:02:46 (1347026566) LustreError: 24899:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 24899:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 24899:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 24899:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 25079:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 25079:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:04:33 (1347026673) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 07:05:44 (1347026744) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 25364:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347026753/real 1347026753] req@ffff880031bdb378 x1412424955991980/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347026759 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 25364:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:07:28 (1347026848) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 07:08:40 (1347026920) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 26037:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 26037:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:10:19 (1347027019) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 07:11:30 (1347027090) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:13:13 (1347027193) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 07:14:25 (1347027265) LustreError: 26765:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 26765:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 26765:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 26765:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 26946:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 26946:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:16:11 (1347027371) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 07:17:22 (1347027442) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 27232:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347027451/real 1347027451] req@ffff88002892d178 x1412424956023447/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347027457 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 27232:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:19:04 (1347027544) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 07:20:16 (1347027616) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 27904:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 27904:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:22:01 (1347027721) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 07:23:13 (1347027793) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:25:05 (1347027905) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 120 sec == 07:26:17 (1347027977) LustreError: 28630:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 28630:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 28630:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 28630:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 28810:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 28810:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:27:56 (1347028076) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 07:29:08 (1347028148) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 29096:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347028157/real 1347028157] req@ffff88002a1570f8 x1412424956054921/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347028163 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 29096:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:30:52 (1347028252) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 07:32:03 (1347028323) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 29770:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 29770:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:33:50 (1347028430) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 07:35:01 (1347028501) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:36:45 (1347028605) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 07:37:57 (1347028677) LustreError: 30495:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 30495:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 30495:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 30495:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 30675:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 30675:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:39:37 (1347028777) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 07:40:49 (1347028849) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 30962:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347028858/real 1347028858] req@ffff88002870ad20 x1412424956086384/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347028864 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 30962:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:42:30 (1347028950) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 07:43:42 (1347029022) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 31634:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 31634:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:45:27 (1347029127) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 07:46:40 (1347029200) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:48:21 (1347029301) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 07:49:33 (1347029373) LustreError: 32359:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 32359:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 32359:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 32359:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 32542:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 32542:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:51:17 (1347029477) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 07:52:31 (1347029551) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 361:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347029560/real 1347029560] req@ffff8800250ed0b8 x1412424956117853/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347029566 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 361:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:54:14 (1347029654) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 07:55:26 (1347029726) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 1046:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 1046:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:57:13 (1347029833) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 07:58:15 (1347029895) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 07:59:58 (1347029998) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 08:01:09 (1347030069) LustreError: 1843:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 1843:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 1843:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 1843:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 2023:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 2023:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:02:50 (1347030170) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 08:04:03 (1347030243) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 2311:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347030248/real 1347030248] req@ffff88002f0a6d20 x1412424956149321/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347030254 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 2311:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:05:42 (1347030342) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 08:06:55 (1347030415) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 2984:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 2984:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:08:34 (1347030514) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 08:09:46 (1347030586) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:11:30 (1347030690) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 08:12:42 (1347030762) LustreError: 3712:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3712:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 3712:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 3712:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 3892:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 3892:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:14:30 (1347030870) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 08:15:41 (1347030941) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 4177:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347030950/real 1347030950] req@ffff880026bd50b8 x1412424956180810/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347030956 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 4177:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:17:20 (1347031040) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 08:18:32 (1347031112) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 4850:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 4850:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:20:19 (1347031219) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 08:21:31 (1347031291) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:23:17 (1347031397) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 08:24:29 (1347031469) LustreError: 5576:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 5576:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 5576:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 5576:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 5756:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5756:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:26:10 (1347031570) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 08:27:23 (1347031643) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 6043:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347031652/real 1347031652] req@ffff880028ee9378 x1412424956212292/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347031658 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 6043:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:29:05 (1347031745) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 08:30:16 (1347031816) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 6715:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 6715:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:32:02 (1347031922) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 08:33:12 (1347031992) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:34:57 (1347032097) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 08:36:09 (1347032169) LustreError: 7441:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 7441:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 7441:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 7441:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 7621:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 7621:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:37:55 (1347032275) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 08:39:07 (1347032347) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 7906:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347032356/real 1347032356] req@ffff880024cd8d60 x1412424956243780/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347032362 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 7906:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:40:52 (1347032452) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 08:42:06 (1347032526) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 8581:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 8581:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:43:52 (1347032632) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 08:45:04 (1347032704) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:46:49 (1347032809) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 08:48:00 (1347032880) LustreError: 9306:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 9306:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 9306:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 9306:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 9486:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 9486:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:49:48 (1347032988) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 08:50:58 (1347033058) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 9773:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347033067/real 1347033067] req@ffff880031bb9378 x1412424956275286/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347033073 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 9773:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:52:38 (1347033158) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 08:53:50 (1347033230) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 10445:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 10445:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:55:37 (1347033337) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 08:56:48 (1347033408) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 08:58:32 (1347033512) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 08:59:44 (1347033584) LustreError: 11170:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 11170:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 11170:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 11170:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 11370:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 11370:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:01:29 (1347033689) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 09:02:41 (1347033761) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 11656:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347033771/real 1347033771] req@ffff88003140ed60 x1412424956306779/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347033777 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 11656:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:04:25 (1347033865) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 09:05:37 (1347033937) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 12327:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 12327:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:07:24 (1347034044) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 09:08:35 (1347034115) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:10:14 (1347034214) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 09:11:26 (1347034286) LustreError: 13055:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 13055:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 13055:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 13055:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 13235:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 13235:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:13:11 (1347034391) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 09:14:22 (1347034462) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 13522:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347034472/real 1347034472] req@ffff88002a4b2d20 x1412424956338240/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347034478 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 13522:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:16:08 (1347034568) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 09:17:19 (1347034639) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 14195:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 14195:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:19:05 (1347034745) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 09:20:17 (1347034817) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:22:05 (1347034925) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 09:23:15 (1347034995) LustreError: 14924:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 14924:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 14924:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 14924:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 15104:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 15104:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:25:02 (1347035102) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 09:26:14 (1347035174) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 15390:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347035180/real 1347035180] req@ffff880028147178 x1412424956369679/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347035186 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 15390:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:27:56 (1347035276) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 09:29:08 (1347035348) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 16064:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 16064:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 3 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:30:54 (1347035454) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 09:32:06 (1347035526) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:34:02 (1347035642) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 09:35:09 (1347035709) LustreError: 16791:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 16791:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 16791:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 16791:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 16971:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 16971:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:36:53 (1347035813) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 09:38:05 (1347035885) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 17256:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347035895/real 1347035895] req@ffff880027d29178 x1412424956401157/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347035901 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 17256:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:39:45 (1347035985) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 09:40:56 (1347036056) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 17929:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 17929:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:42:44 (1347036164) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 09:43:54 (1347036234) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:45:38 (1347036338) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 09:46:50 (1347036410) LustreError: 18654:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 18654:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 18654:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 18654:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 18834:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 18834:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:48:34 (1347036514) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 09:49:47 (1347036587) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 19120:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347036592/real 1347036592] req@ffff880024ec3650 x1412424956432621/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347036598 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 19120:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:51:30 (1347036690) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 09:52:44 (1347036764) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 19794:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 19794:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:54:22 (1347036862) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 09:55:36 (1347036936) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 09:57:14 (1347037034) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 09:58:28 (1347037108) LustreError: 20522:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 20522:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 20522:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 20522:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 20702:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 20702:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:00:16 (1347037216) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 10:01:26 (1347037286) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 21007:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347037296/real 1347037296] req@ffff8800243f51b8 x1412424956464133/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347037302 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 21007:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:03:07 (1347037387) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 10:04:21 (1347037461) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 21678:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 21678:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:06:06 (1347037566) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 10:07:17 (1347037637) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:09:05 (1347037745) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 10:10:18 (1347037818) LustreError: 22405:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 22405:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 22405:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 22405:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 22585:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 22585:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:12:01 (1347037921) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 10:13:15 (1347037995) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 22872:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347038000/real 1347038000] req@ffff88002817f238 x1412424956495595/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347038006 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 22872:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:14:58 (1347038098) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 10:16:11 (1347038171) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 23545:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 23545:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:17:56 (1347038276) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 10:19:09 (1347038349) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:20:51 (1347038451) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 136 sec == 10:22:26 (1347038546) LustreError: 24274:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 24274:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 24274:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 24274:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 24454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 24454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:24:15 (1347038655) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 10:25:28 (1347038728) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 24739:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347038734/real 1347038734] req@ffff880029e0b2f8 x1412424956527110/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347038740 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 24739:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:27:07 (1347038827) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 10:28:22 (1347038902) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 25412:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 25412:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:30:08 (1347039008) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 10:31:22 (1347039082) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:33:05 (1347039185) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 10:34:18 (1347039258) LustreError: 26140:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 26140:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 26140:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 26140:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 26320:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 26320:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:36:06 (1347039366) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 10:37:21 (1347039441) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 26608:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347039451/real 1347039451] req@ffff880029c4f750 x1412424956558596/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347039457 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 26608:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:39:07 (1347039547) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 10:40:20 (1347039620) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 27280:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 27280:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:42:00 (1347039720) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 10:43:14 (1347039794) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:44:54 (1347039894) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 10:46:06 (1347039966) LustreError: 28006:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 28006:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 28006:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 28006:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 28186:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 28186:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:47:50 (1347040070) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 10:49:04 (1347040144) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 28471:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347040149/real 1347040149] req@ffff880028ab9378 x1412424956590060/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347040155 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 28471:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:50:47 (1347040247) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 10:52:01 (1347040321) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 29145:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 29145:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:53:40 (1347040420) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 10:54:54 (1347040494) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:56:46 (1347040606) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 122 sec == 10:57:57 (1347040677) LustreError: 29873:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 29873:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 29873:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 29873:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 30053:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 30053:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 10:59:48 (1347040788) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 122 sec == 11:01:03 (1347040863) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 30359:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347040873/real 1347040873] req@ffff8800288361d8 x1412424956621541/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347040879 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 30359:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:02:48 (1347040968) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 11:04:03 (1347041043) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 31032:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 31032:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:05:55 (1347041155) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 123 sec == 11:07:11 (1347041231) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:08:53 (1347041333) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 11:10:07 (1347041407) LustreError: 31760:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 31760:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 31760:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 31760:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 31941:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 31941:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:11:49 (1347041509) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 11:13:02 (1347041582) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 32228:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347041588/real 1347041588] req@ffff88002611b1b8 x1412424956653017/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347041594 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 32228:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:14:45 (1347041685) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 11:16:00 (1347041760) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 434:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 434:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:17:46 (1347041866) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 11:18:59 (1347041939) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:20:41 (1347042041) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 11:21:57 (1347042117) LustreError: 1183:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 1183:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 1183:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 1183:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 1378:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 1378:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:23:38 (1347042218) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 11:24:52 (1347042292) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 1691:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347042302/real 1347042302] req@ffff8800299a3278 x1412424956684500/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347042308 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 1691:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:26:35 (1347042395) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 11:27:49 (1347042469) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 2365:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 2365:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:29:30 (1347042570) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 11:30:43 (1347042643) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:32:30 (1347042750) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 121 sec == 11:33:44 (1347042824) LustreError: 3094:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3094:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 3094:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 3094:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 3274:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 3274:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:35:24 (1347042924) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 11:36:38 (1347042998) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 3559:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347043004/real 1347043004] req@ffff880037b45378 x1412424956715954/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347043010 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 3559:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:38:22 (1347043102) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 11:39:37 (1347043177) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 4233:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 4233:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:41:22 (1347043282) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 11:42:35 (1347043355) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:44:15 (1347043455) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 11:45:30 (1347043530) LustreError: 4959:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 4959:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 4959:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 4959:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 5139:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5139:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:47:14 (1347043634) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 11:48:28 (1347043708) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 5425:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347043714/real 1347043714] req@ffff880024df10f8 x1412424956747428/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347043720 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 5425:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:50:07 (1347043807) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 11:51:22 (1347043882) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 6100:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 6100:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:53:09 (1347043989) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 120 sec == 11:54:25 (1347044065) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:56:10 (1347044170) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 11:57:26 (1347044246) LustreError: 6826:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6826:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 6826:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 6826:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 7006:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 7006:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 11:59:08 (1347044348) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 12:00:22 (1347044422) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 7292:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347044432/real 1347044432] req@ffff88003d8b1338 x1412424956778921/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347044438 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 7292:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0001_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:02:07 (1347044527) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 12:03:21 (1347044601) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 7983:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 7983:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:05:09 (1347044709) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 120 sec == 12:06:22 (1347044782) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:08:05 (1347044885) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 12:09:19 (1347044959) LustreError: 8708:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 8708:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 8708:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 8708:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 8890:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 8890:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:11:01 (1347045061) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 12:12:15 (1347045135) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 9177:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347045141/real 1347045141] req@ffff880024c0f1b8 x1412424956810403/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347045147 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 9177:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:13:53 (1347045233) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 12:15:01 (1347045301) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 9849:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 9849:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:16:46 (1347045406) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 12:17:58 (1347045478) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:19:40 (1347045580) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 12:20:51 (1347045651) LustreError: 10576:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 10576:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 10576:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 10576:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 10756:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 10756:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:22:39 (1347045759) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 12:23:49 (1347045829) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 11042:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347045839/real 1347045839] req@ffff8800277a50f8 x1412424956841843/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347045845 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 11042:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:25:31 (1347045931) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 12:26:43 (1347046003) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 11713:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 11713:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:28:26 (1347046106) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 12:29:38 (1347046178) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:31:30 (1347046290) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 12:32:41 (1347046361) LustreError: 12441:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 12441:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 12441:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 12441:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 12622:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 12622:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:34:30 (1347046470) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 12:35:43 (1347046543) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 12908:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347046549/real 1347046549] req@ffff880024c0ede0 x1412424956873303/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347046555 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 12908:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:37:24 (1347046644) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 12:38:35 (1347046715) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 13581:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 13581:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:40:22 (1347046822) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 12:41:33 (1347046893) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:43:25 (1347047005) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 12:44:37 (1347047077) LustreError: 14309:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 14309:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 14309:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 14309:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 14489:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 14489:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:46:30 (1347047190) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 124 sec == 12:47:43 (1347047263) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 14775:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347047269/real 1347047269] req@ffff8800251b11f8 x1412424956904793/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347047275 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 14775:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:49:20 (1347047360) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 12:50:32 (1347047432) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 15448:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 15448:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:52:14 (1347047534) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 12:53:26 (1347047606) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:55:17 (1347047717) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 12:56:29 (1347047789) LustreError: 16175:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 16175:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 16175:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 16175:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 16355:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 16355:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 12:58:13 (1347047893) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 12:59:25 (1347047965) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 16641:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347047975/real 1347047975] req@ffff88002ee911f8 x1412424956936273/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347047981 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 16641:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:01:08 (1347048068) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 13:02:21 (1347048141) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 17334:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 17334:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:04:06 (1347048246) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 13:05:18 (1347048318) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:07:01 (1347048421) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 13:08:13 (1347048493) LustreError: 18059:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 18059:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 18059:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 18059:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 18240:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 18240:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:09:58 (1347048598) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 13:11:10 (1347048670) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 18528:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347048680/real 1347048680] req@ffff8800271e2d20 x1412424956967752/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347048686 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 18528:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:12:53 (1347048773) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:14:04 (1347048844) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 19199:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 19199:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:15:47 (1347048947) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:17:00 (1347049020) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:18:44 (1347049124) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 13:19:56 (1347049196) LustreError: 19925:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 19925:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 19925:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 19925:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 20107:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 20107:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:21:37 (1347049297) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 13:22:49 (1347049369) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 20392:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347049379/real 1347049379] req@ffff880029d4f2f8 x1412424956999210/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347049385 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 20392:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:24:33 (1347049473) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:25:45 (1347049545) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 21064:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 21064:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:27:33 (1347049653) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 13:28:45 (1347049725) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:30:36 (1347049836) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 13:31:47 (1347049907) LustreError: 21792:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 21792:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 21792:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 21792:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 21972:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 21972:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:33:28 (1347050008) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:34:41 (1347050081) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 22259:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347050087/real 1347050087] req@ffff8800250ccf20 x1412424957030684/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347050093 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 22259:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:36:20 (1347050180) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:37:33 (1347050253) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 22931:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 22931:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:39:12 (1347050352) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 13:40:24 (1347050424) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:42:13 (1347050533) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 13:43:24 (1347050604) LustreError: 23657:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 23657:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 23657:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 23657:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 23837:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 23837:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:45:11 (1347050711) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 13:46:23 (1347050783) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 24123:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347050794/real 1347050794] req@ffff88003c9ce6f0 x1412424957062162/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347050800 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 24123:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:48:05 (1347050885) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:49:18 (1347050958) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 24798:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 24798:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:51:04 (1347051064) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 13:52:17 (1347051137) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:54:02 (1347051242) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:55:13 (1347051313) LustreError: 25524:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 25524:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 25524:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 25524:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 25704:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 25704:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:56:58 (1347051418) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 13:58:09 (1347051489) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 25989:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347051499/real 1347051499] req@ffff8800264171b8 x1412424957093630/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347051505 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 25989:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 13:59:53 (1347051593) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 14:01:06 (1347051666) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 26683:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 26683:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:02:50 (1347051770) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 14:04:02 (1347051842) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:05:48 (1347051948) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 14:07:00 (1347052020) LustreError: 27408:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 27408:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 27408:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 27408:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 27588:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 27588:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:08:43 (1347052123) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 14:09:55 (1347052195) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 27876:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347052205/real 1347052205] req@ffff880024ea0da0 x1412424957125087/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347052211 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 27876:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:11:38 (1347052298) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 14:12:50 (1347052370) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 28549:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 28549:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:14:32 (1347052472) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 14:15:45 (1347052545) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:17:20 (1347052640) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 14:18:31 (1347052711) LustreError: 29274:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 29274:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 29274:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 29274:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 29454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 29454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:20:09 (1347052809) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 14:21:23 (1347052883) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 29742:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347052889/real 1347052889] req@ffff88002a9d50b8 x1412424957156551/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347052895 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 29742:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:23:05 (1347052985) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 14:24:15 (1347053055) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 30413:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 30413:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:25:58 (1347053158) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 14:27:11 (1347053231) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:28:54 (1347053334) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 14:30:07 (1347053407) LustreError: 31141:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 31141:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 31141:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 31141:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 31322:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 31322:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:31:48 (1347053508) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 14:32:59 (1347053579) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 31607:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347053590/real 1347053590] req@ffff880026196d60 x1412424957188026/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347053596 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 31607:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:34:40 (1347053680) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 14:35:52 (1347053752) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 32280:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 32280:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:37:43 (1347053863) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 119 sec == 14:38:54 (1347053934) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:40:36 (1347054036) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 14:41:47 (1347054107) LustreError: 542:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 542:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 542:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 542:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 727:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 727:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:43:31 (1347054211) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 14:44:43 (1347054283) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 1020:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347054294/real 1347054294] req@ffff8800261970b8 x1412424957219513/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347054300 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 1020:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:46:30 (1347054390) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 14:47:43 (1347054463) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 1743:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 1743:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:49:26 (1347054566) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 14:50:38 (1347054638) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:52:21 (1347054741) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 14:53:32 (1347054812) LustreError: 2472:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 2472:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 2472:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 2472:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 2654:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 2654:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:55:20 (1347054920) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 14:56:33 (1347054993) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 2940:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347055003/real 1347055003] req@ffff880028066da0 x1412424957250988/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347055009 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 2940:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 14:58:16 (1347055096) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 14:59:27 (1347055167) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 3615:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 3615:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:01:12 (1347055272) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 15:02:22 (1347055342) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:04:10 (1347055450) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 15:05:22 (1347055522) LustreError: 4359:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 4359:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 4359:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 4359:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 4539:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 4539:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:07:05 (1347055625) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 15:08:17 (1347055697) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 4825:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347055704/real 1347055704] req@ffff88002a0929c8 x1412424957282433/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347055710 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 4825:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:10:01 (1347055801) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 15:11:14 (1347055874) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 5499:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 5499:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:13:05 (1347055985) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 15:14:14 (1347056054) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:16:01 (1347056161) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 15:17:12 (1347056232) LustreError: 6225:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6225:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 6225:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 6225:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 6405:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 6405:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:18:54 (1347056334) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 15:20:03 (1347056403) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 6694:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347056414/real 1347056414] req@ffff880026e5b0f8 x1412424957313913/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347056420 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 6694:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:21:52 (1347056512) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 15:22:59 (1347056579) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 7365:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 7365:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:24:37 (1347056677) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 15:25:44 (1347056744) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:27:28 (1347056848) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 15:28:36 (1347056916) LustreError: 8091:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 8091:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 8091:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 8091:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 8270:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 8270:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:30:22 (1347057022) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 15:31:33 (1347057093) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 8558:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347057103/real 1347057103] req@ffff8800263051b8 x1412424957345376/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347057109 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 8558:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:33:17 (1347057197) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 15:34:28 (1347057268) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 9230:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 9230:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:36:14 (1347057374) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 15:37:26 (1347057446) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:39:12 (1347057552) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 15:40:24 (1347057624) LustreError: 9956:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 9956:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 9956:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 9956:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 10136:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 10136:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:42:05 (1347057725) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 15:43:16 (1347057796) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 10422:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347057807/real 1347057807] req@ffff8800263050f8 x1412424957376850/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347057813 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 10422:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:45:01 (1347057901) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 15:46:14 (1347057974) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 11094:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 11094:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:47:58 (1347058078) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 15:49:09 (1347058149) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:50:52 (1347058252) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 15:52:05 (1347058325) LustreError: 11822:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 11822:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 11822:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 11822:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 12002:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 12002:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:53:46 (1347058426) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 15:54:57 (1347058497) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 12288:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347058508/real 1347058508] req@ffff880028e36ce0 x1412424957408319/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347058514 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 12288:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:56:41 (1347058601) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 15:57:53 (1347058673) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 12959:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 12959:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 15:59:40 (1347058780) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 16:00:53 (1347058853) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:02:37 (1347058957) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 16:03:49 (1347059029) LustreError: 13704:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 13704:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 13704:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 13704:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 13884:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 13884:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:05:33 (1347059133) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 16:06:44 (1347059204) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 14169:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347059215/real 1347059215] req@ffff8800288372f8 x1412424957439799/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347059221 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 14169:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:08:27 (1347059307) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 16:09:38 (1347059378) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 14849:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 14849:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:11:29 (1347059489) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 118 sec == 16:12:39 (1347059559) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:14:25 (1347059665) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 16:15:37 (1347059737) LustreError: 15579:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 15579:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 15579:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 15579:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 15759:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 15759:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:17:24 (1347059844) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 16:18:36 (1347059916) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 16047:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347059926/real 1347059926] req@ffff88002714ed60 x1412424957471274/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347059932 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 16047:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:20:16 (1347060016) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 16:21:27 (1347060087) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 16722:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 16722:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:23:17 (1347060197) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 16:24:28 (1347060268) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:26:17 (1347060377) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 16:27:28 (1347060448) LustreError: 17448:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 17448:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 17448:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 17448:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 17628:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 17628:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:29:12 (1347060552) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 16:30:23 (1347060623) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 17915:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347060634/real 1347060634] req@ffff880024c391b8 x1412424957502742/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347060640 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 17915:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:32:03 (1347060723) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 16:33:13 (1347060793) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 18586:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 18586:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:34:57 (1347060897) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 16:36:10 (1347060970) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:37:48 (1347061068) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 16:39:02 (1347061142) LustreError: 19313:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 19313:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 19313:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 19313:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 19495:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 19495:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:40:44 (1347061244) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 16:41:56 (1347061316) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 19780:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347061327/real 1347061327] req@ffff8800292d4d60 x1412424957534224/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347061333 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 19780:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:43:35 (1347061415) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 16:44:47 (1347061487) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 20453:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 20453:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:46:32 (1347061592) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 16:47:43 (1347061663) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:49:28 (1347061768) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 16:50:39 (1347061839) LustreError: 21179:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 21179:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 21179:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 21179:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 21360:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 21360:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:52:23 (1347061943) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 16:53:34 (1347062014) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 21645:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347062025/real 1347062025] req@ffff88002524d690 x1412424957565702/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347062031 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 21645:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:55:16 (1347062116) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 16:56:27 (1347062187) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 22317:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 22317:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 16:58:10 (1347062290) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 16:59:19 (1347062359) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:01:04 (1347062464) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 17:02:17 (1347062537) LustreError: 23062:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 23062:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 23062:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 23062:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 23243:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 23243:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:04:00 (1347062640) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 17:05:12 (1347062712) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 23528:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347062723/real 1347062723] req@ffff88002a0932b8 x1412424957597152/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347062729 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 23528:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:06:56 (1347062816) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 17:08:07 (1347062887) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 24201:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 24201:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:09:53 (1347062993) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 17:11:04 (1347063064) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:12:48 (1347063168) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 17:13:58 (1347063238) LustreError: 24929:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 24929:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 24929:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 24929:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 25109:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 25109:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:15:45 (1347063345) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 17:16:52 (1347063412) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 25395:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347063419/real 1347063419] req@ffff88002402aca0 x1412424957628595/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347063425 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 25395:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:18:33 (1347063513) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 103 sec == 17:19:36 (1347063576) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 26069:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 26069:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:21:17 (1347063677) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 17:22:25 (1347063745) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:24:03 (1347063843) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 17:25:11 (1347063911) LustreError: 26795:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 26795:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 26795:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 26795:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 26975:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 26975:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:26:54 (1347064014) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 17:28:02 (1347064082) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 27260:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347064089/real 1347064089] req@ffff88002ab2aca0 x1412424957660019/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347064095 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 27260:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:29:43 (1347064183) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 17:30:51 (1347064251) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 27933:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 27933:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:32:29 (1347064349) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 17:33:37 (1347064417) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:35:15 (1347064515) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 17:36:23 (1347064583) LustreError: 28659:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 28659:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 28659:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 28659:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 28840:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 28840:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:38:04 (1347064684) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 17:39:13 (1347064753) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 29125:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347064759/real 1347064759] req@ffff880027f590f8 x1412424957691435/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347064765 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 29125:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:40:54 (1347064854) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 17:42:01 (1347064921) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 29798:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 29798:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:43:41 (1347065021) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 17:44:49 (1347065089) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:46:24 (1347065184) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 17:47:31 (1347065251) LustreError: 30524:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 30524:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 30524:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 30524:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 30704:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 30704:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:49:12 (1347065352) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 17:50:19 (1347065419) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 30990:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347065430/real 1347065430] req@ffff880027a3cd20 x1412424957722845/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347065436 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 30990:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:52:02 (1347065522) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 17:53:09 (1347065589) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 31662:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 31662:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:54:54 (1347065694) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 17:56:02 (1347065762) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 17:57:47 (1347065867) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 17:58:53 (1347065933) LustreError: 32388:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 32388:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 32388:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 32388:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 32571:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 32571:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:00:34 (1347066034) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 18:01:41 (1347066101) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 407:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347066108/real 1347066108] req@ffff88002896b2b8 x1412424957754266/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347066114 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 407:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:03:20 (1347066200) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 18:04:27 (1347066267) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 1095:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 1095:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:06:04 (1347066364) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 18:07:11 (1347066431) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:08:51 (1347066531) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 18:10:00 (1347066600) LustreError: 1870:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 1870:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 1870:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 1870:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 2050:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 2050:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:11:43 (1347066703) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 18:12:51 (1347066771) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 2337:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347066778/real 1347066778] req@ffff88002aa270f8 x1412424957785706/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347066784 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 2337:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:14:27 (1347066867) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 18:15:36 (1347066936) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 3009:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 3009:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:17:23 (1347067043) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 18:18:32 (1347067112) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:20:06 (1347067206) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 18:21:15 (1347067275) LustreError: 3737:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3737:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 3737:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 3737:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 3917:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 3917:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:22:54 (1347067374) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 18:24:02 (1347067442) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 4203:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347067448/real 1347067448] req@ffff8800266b9710 x1412424957817123/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347067454 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 4203:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:25:40 (1347067540) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 18:26:50 (1347067610) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 4875:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 4875:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:28:30 (1347067710) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 18:29:37 (1347067777) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:31:16 (1347067876) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 18:32:23 (1347067943) LustreError: 5603:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 5603:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 5603:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 5603:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 5782:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5782:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:34:00 (1347068040) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 18:35:08 (1347068108) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 6068:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347068115/real 1347068115] req@ffff880026567b28 x1412424957848526/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347068121 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 6068:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:36:46 (1347068206) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 18:37:52 (1347068272) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 6739:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 6739:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:39:35 (1347068375) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 18:40:43 (1347068443) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:42:24 (1347068544) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 18:43:31 (1347068611) LustreError: 7465:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 7465:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 7465:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 7465:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 7645:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 7645:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:45:12 (1347068712) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 18:46:19 (1347068779) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 7931:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347068786/real 1347068786] req@ffff8800250ae988 x1412424957879942/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347068792 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 7931:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:47:58 (1347068878) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 18:49:04 (1347068944) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 8604:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 8604:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:50:45 (1347069045) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 18:51:51 (1347069111) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:53:43 (1347069223) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 18:54:51 (1347069291) LustreError: 9330:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 9330:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 9330:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 9330:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 9510:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 9510:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:56:38 (1347069398) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 18:57:45 (1347069465) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 9795:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347069472/real 1347069472] req@ffff8800256b8ca0 x1412424957911364/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347069478 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 9795:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 18:59:26 (1347069566) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 19:00:33 (1347069633) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 10486:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 10486:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:02:13 (1347069733) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 19:03:21 (1347069801) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:05:00 (1347069900) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 19:06:07 (1347069967) LustreError: 11214:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 11214:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 11214:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 11214:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 11394:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 11394:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:07:48 (1347070068) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 19:08:56 (1347070136) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 11679:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347070144/real 1347070144] req@ffff880031860ca0 x1412424957942779/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347070150 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 11679:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:10:35 (1347070235) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 19:11:42 (1347070302) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 12353:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 12353:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:13:21 (1347070401) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 19:14:28 (1347070468) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:16:07 (1347070567) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 19:17:16 (1347070636) LustreError: 13078:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 13078:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 13078:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 13078:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 13258:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 13258:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:18:58 (1347070738) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 19:20:05 (1347070805) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 13545:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347070812/real 1347070812] req@ffff88003ca712f8 x1412424957974181/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347070818 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 13545:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:21:54 (1347070914) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 19:23:01 (1347070981) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 14216:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 14216:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:24:43 (1347071083) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 19:25:50 (1347071150) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:27:31 (1347071251) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 19:28:37 (1347071317) LustreError: 14942:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 14942:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 14942:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 14942:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 15122:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 15122:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:30:23 (1347071423) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 19:31:31 (1347071491) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 15411:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347071498/real 1347071498] req@ffff880026727650 x1412424958005611/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347071504 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 15411:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:33:09 (1347071589) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 19:34:17 (1347071657) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 16082:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 16082:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:35:59 (1347071759) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 19:37:07 (1347071827) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:38:42 (1347071922) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 19:39:50 (1347071990) LustreError: 16808:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 16808:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 16808:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 16808:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 16989:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 16989:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:41:27 (1347072087) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 19:42:33 (1347072153) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 17275:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347072165/real 1347072165] req@ffff880025642630 x1412424958037023/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347072171 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 17275:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:44:16 (1347072256) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 19:45:25 (1347072325) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 17946:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 17946:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:47:03 (1347072423) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 19:48:11 (1347072491) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:49:54 (1347072594) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 19:51:02 (1347072662) LustreError: 18675:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 18675:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 18675:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 18675:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 18855:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 18855:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:52:40 (1347072760) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 19:53:47 (1347072827) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 19142:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347072834/real 1347072834] req@ffff88002ee912f8 x1412424958068411/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347072840 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 19142:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:55:29 (1347072929) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 19:56:36 (1347072996) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 19813:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 19813:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 19:58:16 (1347073096) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 19:59:23 (1347073163) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:01:05 (1347073265) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 20:02:12 (1347073332) LustreError: 20558:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 20558:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 20558:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 20558:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 20738:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 20738:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:03:48 (1347073428) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 20:04:56 (1347073496) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 21024:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347073503/real 1347073503] req@ffff880026a3f138 x1412424958099821/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347073509 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 21024:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:06:33 (1347073593) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 20:07:39 (1347073659) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 21695:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 21695:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:09:21 (1347073761) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 20:10:28 (1347073828) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:12:07 (1347073927) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 20:13:14 (1347073994) LustreError: 22421:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 22421:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 22421:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 22421:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 22601:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 22601:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:15:02 (1347074102) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 20:16:10 (1347074170) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 22887:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347074178/real 1347074178] req@ffff880029950ce0 x1412424958131226/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347074184 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 22887:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:17:53 (1347074273) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 20:19:00 (1347074340) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 23560:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 23560:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:20:43 (1347074443) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 20:21:50 (1347074510) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:23:34 (1347074614) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 20:24:43 (1347074683) LustreError: 24285:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 24285:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 24285:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 24285:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 24465:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 24465:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:26:23 (1347074783) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 20:27:30 (1347074850) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 24751:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347074861/real 1347074861] req@ffff8800385762d8 x1412424958162664/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347074867 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 24751:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:29:15 (1347074955) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 20:30:23 (1347075023) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 25424:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 25424:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:32:02 (1347075122) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 20:33:10 (1347075190) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:34:47 (1347075287) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 20:35:54 (1347075354) LustreError: 26149:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 26149:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 26149:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 26149:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 26329:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 26329:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:37:35 (1347075455) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 20:38:43 (1347075523) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 26615:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347075531/real 1347075531] req@ffff8800274831b8 x1412424958194086/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347075537 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 26615:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:40:27 (1347075627) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 20:41:34 (1347075694) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 27288:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 27288:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:43:20 (1347075800) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 20:44:27 (1347075867) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:46:06 (1347075966) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 20:47:14 (1347076034) LustreError: 28015:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 28015:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 28015:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 28015:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 28195:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 28195:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:48:52 (1347076132) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 20:50:00 (1347076200) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 28483:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347076207/real 1347076207] req@ffff880025e6f0f8 x1412424958225512/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347076213 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 28483:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:51:33 (1347076293) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 20:52:42 (1347076362) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 29154:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 29154:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:54:29 (1347076469) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 20:55:37 (1347076537) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:57:13 (1347076633) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 20:58:21 (1347076701) LustreError: 29879:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 29879:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 29879:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 29879:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 30059:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 30059:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 20:59:55 (1347076795) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 21:01:02 (1347076862) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 30364:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347076873/real 1347076873] req@ffff88002702ed60 x1412424958256927/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347076879 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 30364:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:02:55 (1347076975) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 21:04:02 (1347077042) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 31036:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 31036:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:05:50 (1347077150) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 21:06:58 (1347077218) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:08:38 (1347077318) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 21:09:46 (1347077386) LustreError: 31761:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 31761:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 31761:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 31761:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 31944:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 31944:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:11:24 (1347077484) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 21:12:30 (1347077550) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 32230:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347077562/real 1347077562] req@ffff88002401d0f8 x1412424958288374/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347077568 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 32230:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:14:16 (1347077656) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 21:15:25 (1347077725) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 435:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 435:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:17:10 (1347077830) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 21:18:18 (1347077898) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:20:00 (1347078000) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 21:21:08 (1347078068) LustreError: 1187:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 1187:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 1187:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 1187:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 1382:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 1382:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:22:47 (1347078167) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 21:23:56 (1347078236) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 1693:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347078248/real 1347078248] req@ffff880029652d60 x1412424958319799/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347078254 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 1693:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:25:46 (1347078346) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 116 sec == 21:26:53 (1347078413) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 2366:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 2366:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:28:33 (1347078513) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 21:29:40 (1347078580) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:31:25 (1347078685) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 21:32:32 (1347078752) LustreError: 3095:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3095:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 3095:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 3095:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 3275:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 3275:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:34:11 (1347078851) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 21:35:18 (1347078918) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 3560:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347078926/real 1347078926] req@ffff880029b7ed60 x1412424958351241/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347078932 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 3560:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:36:59 (1347079019) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 21:38:07 (1347079087) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 4231:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 4231:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:39:47 (1347079187) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 21:40:54 (1347079254) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:42:32 (1347079352) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 21:43:39 (1347079419) LustreError: 4959:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 4959:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 4959:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 4959:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 5139:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5139:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:45:28 (1347079528) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 21:46:36 (1347079596) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 5425:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347079604/real 1347079604] req@ffff8800288e2e60 x1412424958382662/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347079610 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 5425:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:48:16 (1347079696) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 21:49:24 (1347079764) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 6098:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 6098:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:51:01 (1347079861) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 21:52:09 (1347079929) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:53:46 (1347080026) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 21:54:53 (1347080093) LustreError: 6824:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6824:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 6824:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 6824:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 7004:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 7004:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:56:37 (1347080197) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 21:57:44 (1347080264) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 7289:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347080275/real 1347080275] req@ffff8800276734d0 x1412424958414073/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347080281 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 7289:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 21:59:26 (1347080366) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 22:00:34 (1347080434) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 7980:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 7980:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:02:09 (1347080529) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 22:03:17 (1347080597) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:04:54 (1347080694) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 22:06:02 (1347080762) LustreError: 8706:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 8706:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 8706:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 8706:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 8886:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 8886:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:07:38 (1347080858) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 22:08:46 (1347080926) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 9172:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347080934/real 1347080934] req@ffff880026bb8570 x1412424958445490/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347080940 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 9172:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:10:30 (1347081030) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 22:11:38 (1347081098) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 9846:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 9846:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:13:18 (1347081198) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 22:14:25 (1347081265) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:15:58 (1347081358) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 22:17:07 (1347081427) LustreError: 10571:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 10571:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 10571:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 10571:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 10751:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 10751:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:18:49 (1347081529) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 22:19:56 (1347081596) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 11039:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347081604/real 1347081604] req@ffff880027990f60 x1412424958476902/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347081610 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 11039:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:21:34 (1347081694) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 22:22:42 (1347081762) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 11710:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 11710:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:24:23 (1347081863) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 22:25:30 (1347081930) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:27:10 (1347082030) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 22:28:18 (1347082098) LustreError: 12435:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 12435:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 12435:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 12435:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 12615:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 12615:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:29:59 (1347082199) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 22:31:06 (1347082266) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 12903:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347082274/real 1347082274] req@ffff88002a9d4ea0 x1412424958508311/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347082280 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 12903:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:32:43 (1347082363) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 22:33:50 (1347082430) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 13574:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 13574:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:35:34 (1347082534) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 22:36:43 (1347082603) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:38:24 (1347082704) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 22:39:31 (1347082771) LustreError: 14300:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 14300:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 14300:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 14300:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 14482:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 14482:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:41:11 (1347082871) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 22:42:18 (1347082938) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 14767:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347082950/real 1347082950] req@ffff88003c95d0f8 x1412424958539737/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347082956 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 14767:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:44:04 (1347083044) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 22:45:12 (1347083112) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 15440:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 15440:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:46:53 (1347083213) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 22:48:01 (1347083281) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:49:39 (1347083379) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 22:50:46 (1347083446) LustreError: 16167:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 16167:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 16167:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 16167:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 16348:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 16348:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:52:31 (1347083551) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 22:53:38 (1347083618) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 16634:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347083626/real 1347083626] req@ffff88002a156de0 x1412424958571168/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347083632 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 16634:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:55:16 (1347083716) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 22:56:24 (1347083784) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 17306:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 17306:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 22:58:01 (1347083881) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 22:59:09 (1347083949) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:00:46 (1347084046) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 23:01:55 (1347084115) LustreError: 18051:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 18051:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 18051:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 18051:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 18231:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 18231:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:03:34 (1347084214) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 23:04:41 (1347084281) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 18518:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347084289/real 1347084289] req@ffff880027b29278 x1412424958602584/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347084295 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 18518:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:06:19 (1347084379) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 23:07:28 (1347084448) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 19190:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 19190:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:09:03 (1347084543) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 103 sec == 23:10:09 (1347084609) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:12:02 (1347084722) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 23:13:09 (1347084789) LustreError: 19916:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 19916:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 19916:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 19916:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 20096:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 20096:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:14:49 (1347084889) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 23:15:58 (1347084958) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 20382:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347084965/real 1347084965] req@ffff8800253a50f8 x1412424958634017/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347084971 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 20382:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:17:36 (1347085056) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 23:18:44 (1347085124) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 21057:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 21057:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:20:26 (1347085226) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 23:21:34 (1347085294) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:23:11 (1347085391) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 23:24:18 (1347085458) LustreError: 21781:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 21781:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 21781:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 21781:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 21961:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 21961:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:25:57 (1347085557) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 23:27:06 (1347085626) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 22248:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347085638/real 1347085638] req@ffff880027c9cca0 x1412424958665458/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347085644 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 22248:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:28:48 (1347085728) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 23:29:56 (1347085796) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 22922:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 22922:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:31:36 (1347085896) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 23:32:44 (1347085964) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:34:23 (1347086063) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 23:35:31 (1347086131) LustreError: 23646:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 23646:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 23646:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 23646:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 23826:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 23826:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:37:10 (1347086230) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 23:38:17 (1347086297) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 24113:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347086306/real 1347086306] req@ffff88002ee78ee0 x1412424958696889/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347086312 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 24113:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:40:00 (1347086400) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 23:41:08 (1347086468) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 24787:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 24787:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:42:53 (1347086573) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 23:44:00 (1347086640) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:45:46 (1347086746) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 23:46:54 (1347086814) LustreError: 25512:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 25512:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 25512:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 25512:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 25692:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 25692:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:48:30 (1347086910) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 23:49:37 (1347086977) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 25978:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347086985/real 1347086985] req@ffff88002964d0b8 x1412424958728353/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347086991 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 25978:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:51:17 (1347087077) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 23:52:25 (1347087145) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 26661:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 26661:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:54:04 (1347087244) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 23:55:11 (1347087311) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:56:49 (1347087409) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 23:57:56 (1347087476) LustreError: 27385:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 27385:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 27385:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 27385:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 27565:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 27565:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 23:59:40 (1347087580) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 00:00:49 (1347087649) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 27853:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347087657/real 1347087657] req@ffff880026590ee0 x1412424958759769/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347087663 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 27853:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:02:28 (1347087748) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 00:03:37 (1347087817) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 28547:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 28547:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:05:12 (1347087912) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 00:06:20 (1347087980) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:07:57 (1347088077) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 00:09:04 (1347088144) LustreError: 29272:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 29272:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 29272:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 29272:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 29454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 29454:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:10:41 (1347088241) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 00:11:49 (1347088309) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 29741:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347088317/real 1347088317] req@ffff880025b6b0f8 x1412424958791172/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347088323 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 29741:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:13:27 (1347088407) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 00:14:34 (1347088474) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 30413:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 30413:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:16:15 (1347088575) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 00:17:23 (1347088643) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:19:02 (1347088742) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 00:20:10 (1347088810) LustreError: 31139:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 31139:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 31139:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 31139:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 31319:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 31319:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:21:51 (1347088911) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 00:22:58 (1347088978) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 31604:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347088986/real 1347088986] req@ffff880025288ea0 x1412424958822585/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347088992 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 31604:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:24:38 (1347089078) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 00:25:45 (1347089145) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 32276:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 32276:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:27:28 (1347089248) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 00:28:36 (1347089316) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:30:18 (1347089418) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 00:31:26 (1347089486) LustreError: 539:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 539:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 539:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 539:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 724:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 724:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:33:04 (1347089584) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 00:34:10 (1347089650) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 1016:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347089663/real 1347089663] req@ffff8800253c6ea0 x1412424958853991/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347089669 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 1016:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:35:48 (1347089748) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 00:36:57 (1347089817) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 1739:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 1739:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:38:36 (1347089916) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 00:39:43 (1347089983) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:41:25 (1347090085) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 00:42:31 (1347090151) LustreError: 2469:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 2469:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 2469:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 2469:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 2650:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 2650:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:44:15 (1347090255) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 00:45:22 (1347090322) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 2935:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347090330/real 1347090330] req@ffff880031317238 x1412424958885396/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347090336 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 2935:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:47:01 (1347090421) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 00:48:08 (1347090488) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 3606:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 3606:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:49:49 (1347090589) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 00:50:56 (1347090656) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:52:42 (1347090762) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 115 sec == 00:53:51 (1347090831) LustreError: 4335:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 4335:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 4335:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 4335:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 4515:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 4515:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:55:28 (1347090928) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 00:56:35 (1347090995) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 4800:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347091003/real 1347091003] req@ffff880028d8ace0 x1412424958916804/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347091009 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 4800:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 00:58:18 (1347091098) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 00:59:25 (1347091165) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 5473:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 5473:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:01:02 (1347091262) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 01:02:10 (1347091330) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:03:51 (1347091431) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 01:04:59 (1347091499) LustreError: 6222:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 6222:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 6222:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 6222:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 6403:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 6403:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:06:34 (1347091594) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 01:07:43 (1347091663) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 6688:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347091671/real 1347091671] req@ffff880028fc7238 x1412424958948214/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347091677 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 6688:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:09:22 (1347091762) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 01:10:29 (1347091829) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 7360:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 7360:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:12:11 (1347091931) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 01:13:18 (1347091998) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:14:54 (1347092094) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 01:16:02 (1347092162) LustreError: 8085:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 8085:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 8085:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 8085:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 8265:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 8265:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:17:38 (1347092258) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 01:18:45 (1347092325) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 8551:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347092333/real 1347092333] req@ffff880028a07378 x1412424958979640/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347092339 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 8551:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:20:24 (1347092424) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 01:21:31 (1347092491) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 9224:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 9224:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:23:11 (1347092591) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 01:24:18 (1347092658) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:25:57 (1347092757) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 01:27:05 (1347092825) LustreError: 9949:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 9949:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 9949:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 9949:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 10129:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 10129:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:28:38 (1347092918) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 103 sec == 01:29:44 (1347092984) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 10415:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347092996/real 1347092996] req@ffff880028845278 x1412424959011049/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347093002 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 10415:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:31:26 (1347093086) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 01:32:34 (1347093154) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 11089:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 11089:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:34:18 (1347093258) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 01:35:26 (1347093326) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:37:02 (1347093422) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 01:38:09 (1347093489) LustreError: 11813:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 11813:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 11813:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 11813:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 11994:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 11994:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:39:53 (1347093593) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 01:41:01 (1347093661) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 12281:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347093670/real 1347093670] req@ffff88003ca76530 x1412424959042479/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347093676 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 12281:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:42:39 (1347093759) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 01:43:46 (1347093826) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 12953:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 12953:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:45:29 (1347093929) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 01:46:38 (1347093998) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:48:20 (1347094100) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 01:49:28 (1347094168) LustreError: 13678:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 13678:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 13678:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 13678:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 13859:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 13859:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:51:02 (1347094262) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 103 sec == 01:52:09 (1347094329) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 14145:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347094342/real 1347094342] req@ffff88003dbdad20 x1412424959073897/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347094348 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 14145:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:53:53 (1347094433) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 01:55:00 (1347094500) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 14817:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 14817:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:56:41 (1347094601) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 01:57:46 (1347094666) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 01:59:26 (1347094766) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 02:00:33 (1347094833) LustreError: 15542:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 15542:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 15542:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 15542:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 15745:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 15745:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:02:18 (1347094938) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 02:03:25 (1347095005) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 16031:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347095017/real 1347095017] req@ffff8800284951b8 x1412424959105331/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347095023 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 16031:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:05:08 (1347095108) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 02:06:16 (1347095176) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 16703:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 16703:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:08:03 (1347095283) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 02:09:10 (1347095350) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:10:50 (1347095450) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 02:11:58 (1347095518) LustreError: 17431:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 17431:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 17431:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 17431:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 17611:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 17611:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:13:41 (1347095621) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 02:14:48 (1347095688) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 17898:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347095700/real 1347095700] req@ffff88003da04d20 x1412424959136755/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347095706 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 17898:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:16:35 (1347095795) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 02:17:43 (1347095863) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 18570:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 18570:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:19:22 (1347095962) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 02:20:31 (1347096031) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:22:08 (1347096128) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 02:23:16 (1347096196) LustreError: 19297:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 19297:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 19297:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 19297:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 19477:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 19477:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:24:55 (1347096295) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 02:26:03 (1347096363) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 19763:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347096372/real 1347096372] req@ffff88002ee79138 x1412424959168199/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347096378 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 19763:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:27:38 (1347096458) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 02:28:45 (1347096525) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 20436:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 20436:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:30:28 (1347096628) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:31:37 (1347096697) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:33:15 (1347096795) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 02:34:21 (1347096861) LustreError: 21160:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 21160:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 21160:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 21160:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 21340:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 21340:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:36:03 (1347096963) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 02:37:10 (1347097030) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 21626:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347097039/real 1347097039] req@ffff8800252891b8 x1412424959199623/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347097045 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 21626:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:38:46 (1347097126) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 02:39:54 (1347097194) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 22300:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 22300:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:41:33 (1347097293) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 02:42:40 (1347097360) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:44:20 (1347097460) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 02:45:27 (1347097527) LustreError: 23024:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 23024:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 23024:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 23024:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 23204:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 23204:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:47:10 (1347097630) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 02:48:17 (1347097697) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 23490:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347097709/real 1347097709] req@ffff880028caede0 x1412424959231041/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347097715 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 23490:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:49:58 (1347097798) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 02:51:05 (1347097865) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 24164:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 24164:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -19 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:52:46 (1347097966) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 02:53:53 (1347098033) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:55:30 (1347098130) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 02:56:38 (1347098198) LustreError: 24891:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 24891:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 24891:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 24891:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 25071:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 25071:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 02:58:23 (1347098303) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 02:59:29 (1347098369) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 25357:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347098382/real 1347098382] req@ffff880024e58ee0 x1412424959262435/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347098388 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 25357:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:01:12 (1347098472) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 03:02:19 (1347098539) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 26054:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 26054:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:04:00 (1347098640) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 03:05:08 (1347098708) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:06:50 (1347098810) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 03:07:57 (1347098877) LustreError: 26779:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 26779:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 26779:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 26779:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 26959:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 26959:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:09:37 (1347098977) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 03:10:45 (1347099045) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 27246:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347099054/real 1347099054] req@ffff880024cf6da0 x1412424959293853/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347099060 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 27246:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:12:26 (1347099146) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 03:13:34 (1347099214) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 27919:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 27919:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:15:14 (1347099314) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 03:16:20 (1347099380) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:18:01 (1347099481) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 03:19:08 (1347099548) LustreError: 28643:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 28643:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 28643:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 28643:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 28826:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 28826:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:20:45 (1347099645) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 03:21:53 (1347099713) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 29327:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347099726/real 1347099726] req@ffff88003d55e908 x1412424959325278/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347099732 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 29327:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:23:37 (1347099817) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 03:24:45 (1347099885) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 29999:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 29999:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:26:27 (1347099987) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 03:27:34 (1347100054) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:29:12 (1347100152) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 03:30:20 (1347100220) LustreError: 30725:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 30725:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 30725:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 30725:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 30905:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 30905:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:32:04 (1347100324) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 03:33:12 (1347100392) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 31192:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347100401/real 1347100401] req@ffff88003c955510 x1412424959356715/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347100407 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 31192:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:34:53 (1347100493) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 03:36:00 (1347100560) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 31863:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 31863:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:37:36 (1347100656) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 03:38:43 (1347100723) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:40:24 (1347100824) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 03:41:32 (1347100892) LustreError: 32590:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 32590:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 32590:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 32590:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 303:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 303:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:43:08 (1347100988) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 03:44:15 (1347101055) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 590:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347101064/real 1347101064] req@ffff88003c95d928 x1412424959388125/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347101070 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 590:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:45:50 (1347101150) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 03:46:58 (1347101218) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 1290:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 1290:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:48:37 (1347101317) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 03:49:44 (1347101384) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:51:24 (1347101484) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 03:52:30 (1347101550) LustreError: 2053:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 2053:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 2053:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 2053:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 2235:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 2235:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:54:07 (1347101647) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 03:55:14 (1347101714) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 2521:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347101727/real 1347101727] req@ffff8800241632b8 x1412424959419556/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347101733 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 2521:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:57:00 (1347101820) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 03:58:07 (1347101887) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 3193:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 3193:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 03:59:55 (1347101995) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 114 sec == 04:01:01 (1347102061) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:02:47 (1347102167) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 04:03:54 (1347102234) LustreError: 3941:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 3941:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 3941:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 3941:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 4121:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 4121:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:05:36 (1347102336) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 04:06:44 (1347102404) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 4406:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347102412/real 1347102412] req@ffff8800258ed690 x1412424959451001/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347102418 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 4406:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:08:20 (1347102500) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 04:09:27 (1347102567) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 5079:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 5079:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:11:06 (1347102666) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 04:12:14 (1347102734) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:13:53 (1347102833) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 04:15:00 (1347102900) LustreError: 5806:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 5806:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 5806:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 5806:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 5986:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 5986:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:16:38 (1347102998) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 105 sec == 04:17:46 (1347103066) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 6271:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347103075/real 1347103075] req@ffff88002935aea0 x1412424959482426/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347103081 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 6271:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:19:24 (1347103164) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 04:20:31 (1347103231) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 6944:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 6944:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages LustreError: 11-0: an error occurred while communicating with 192.168.56.105@tcp. The ost_connect operation failed with -11 Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:22:08 (1347103328) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 04:23:15 (1347103395) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:25:00 (1347103500) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 04:26:08 (1347103568) LustreError: 7671:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 7671:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 7671:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 7671:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 7851:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 7851:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:27:48 (1347103668) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 04:28:55 (1347103735) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 8136:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347103748/real 1347103748] req@ffff880024de09c8 x1412424959513860/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347103754 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 8136:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:30:35 (1347103835) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 04:31:43 (1347103903) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 8809:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 8809:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:33:26 (1347104006) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 04:34:33 (1347104073) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:36:11 (1347104171) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 04:37:18 (1347104238) LustreError: 9535:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 9535:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 9535:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 9535:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 9715:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 9715:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:39:00 (1347104340) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 04:40:09 (1347104409) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 10004:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347104418/real 1347104418] req@ffff880025a64908 x1412424959545284/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347104424 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 10004:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:41:56 (1347104516) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 113 sec == 04:43:02 (1347104582) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 10675:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 10675:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:44:43 (1347104683) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 04:45:51 (1347104751) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:47:32 (1347104852) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 04:48:40 (1347104920) LustreError: 11402:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 11402:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 11402:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 11402:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 11582:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 11582:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:50:19 (1347105019) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 04:51:28 (1347105088) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 11869:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347105097/real 1347105097] req@ffff88002a95b0f8 x1412424959576719/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347105103 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 11869:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:53:11 (1347105191) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 04:54:18 (1347105258) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: 12540:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 12540:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:56:03 (1347105363) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 04:57:11 (1347105431) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 04:58:52 (1347105532) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 04:59:58 (1347105598) Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? LustreError: 13268:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 13268:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 13268:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 13268:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 13466:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 13466:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:01:55 (1347105715) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 117 sec == 05:03:03 (1347105783) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 13751:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347105792/real 1347105792] req@ffff880025ca4098 x1412424959608176/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347105798 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 13751:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:04:36 (1347105876) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 103 sec == 05:05:43 (1347105943) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 14424:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 14424:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:07:24 (1347106044) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 05:08:32 (1347106112) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:10:13 (1347106213) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 05:11:21 (1347106281) LustreError: 15151:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 15151:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 15151:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 15151:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 15331:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 15331:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:13:04 (1347106384) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 05:14:11 (1347106451) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 15617:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347106460/real 1347106460] req@ffff880029400908 x1412424959639599/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347106466 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 15617:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:15:55 (1347106555) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 111 sec == 05:17:03 (1347106623) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 16289:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 16289:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:18:41 (1347106721) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 05:19:48 (1347106788) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:21:28 (1347106888) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 05:22:36 (1347106956) LustreError: 17017:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 17017:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 17017:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 17017:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 17197:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 17197:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:24:21 (1347107061) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 05:25:28 (1347107128) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 17482:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347107137/real 1347107137] req@ffff88003da05378 x1412424959671045/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347107143 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 17482:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:27:08 (1347107228) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 05:28:15 (1347107295) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 18154:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 18154:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 4 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:29:54 (1347107394) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 05:31:02 (1347107462) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:32:42 (1347107562) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 05:33:49 (1347107629) LustreError: 18883:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 18883:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 18883:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 18883:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 19063:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 19063:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:35:32 (1347107732) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 109 sec == 05:36:39 (1347107799) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 19348:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347107813/real 1347107813] req@ffff8800257f6ce0 x1412424959702464/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347107819 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 19348:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:38:29 (1347107909) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 05:39:36 (1347107976) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 20022:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 20022:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:41:17 (1347108077) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 05:42:23 (1347108143) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:44:10 (1347108250) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 110 sec == 05:45:18 (1347108318) LustreError: 20747:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 20747:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 20747:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 20747:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 20927:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 20927:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:47:03 (1347108423) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 05:48:11 (1347108491) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 21212:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347108500/real 1347108500] req@ffff8800243972b8 x1412424959733872/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347108506 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 21212:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:49:49 (1347108589) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 107 sec == 05:50:55 (1347108655) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: lustre-MDT0000: Temporarily refusing client connection from 192.168.56.106@tcp Lustre: Skipped 1 previous similar message Lustre: 21886:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 21886:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 5 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:52:42 (1347108762) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 112 sec == 05:53:50 (1347108830) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:55:32 (1347108932) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 05:56:40 (1347109000) LustreError: 22612:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway LustreError: 22612:0:(ldlm_request.c:1166:ldlm_cli_cancel_req()) Skipped 7 previous similar messages LustreError: 22612:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 LustreError: 22612:0:(ldlm_request.c:1792:ldlm_cli_cancel_list()) Skipped 7 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MGC192.168.56.104@tcp: Reactivating import Lustre: Skipped 3 previous similar messages Lustre: lustre-MDT0000: new disk, initializing Lustre: Skipped 3 previous similar messages Lustre: 22792:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) lustre-MDT0000: identity upcall set to /usr/sbin/l_getidentity Lustre: 22792:0:(mdt_lproc.c:418:lprocfs_wr_identity_upcall()) Skipped 7 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Lustre: Skipped 3 previous similar messages Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 05:58:18 (1347109098) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 106 sec == 05:59:24 (1347109164) Lustre: MGS has stopped. Lustre: Skipped 3 previous similar messages Lustre: 23077:0:(client.c:1905:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1347109177/real 1347109177] req@ffff88002950ef20 x1412424959765306/t0(0) o251->MGC192.168.56.104@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1347109183 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 Lustre: 23077:0:(client.c:1905:ptlrpc_expire_one_request()) Skipped 3 previous similar messages Lustre: server umount lustre-MDT0000 complete Lustre: Skipped 3 previous similar messages LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Lustre: Skipped 27 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:01:04 (1347109264) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 104 sec == 06:02:11 (1347109331) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: 23769:0:(mgc_request.c:1534:mgc_process_recover_log()) Process recover log lustre-mdtir error -22 Lustre: 23769:0:(mgc_request.c:1534:mgc_process_recover_log()) Skipped 6 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:03:55 (1347109435) Lustre: DEBUG MARKER: == sanity sanity.sh test complete, duration 108 sec == 06:05:03 (1347109503) LDISKFS-fs (dm-2): warning: maximal mount count reached, running e2fsck is recommended LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=off. Opts: Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Lustre: Skipped 39 previous similar messages Lustre: DEBUG MARKER: Using TIMEOUT=20 Lustre: DEBUG MARKER: only running test 103 Lustre: DEBUG MARKER: excepting tests: 27u 42a 42b 42c 42d 45 51d 68b Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24v 27m 36f 36g 36h 51b 60c 63 64b 68 71 73 77f 78 101a 103 115 120g 124b Lustre: DEBUG MARKER: == sanity test 103: acl test =========================================== 06:06:43 (1347109603) kfree_debugcheck: out of range ptr 6000100000002h. ------------[ cut here ]------------ kernel BUG at mm/slab.c:2911! invalid opcode: 0000 [#1] SMP last sysfs file: /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/PNP0C0A:00/power_supply/BAT0/energy_full CPU 0 Modules linked in: cmm(U) osd_ldiskfs(U) mdt(U) mdd(U) mds(U) fsfilt_ldiskfs(U) exportfs mgs(U) mgc(U) ldiskfs(U) lustre(U) lquota(U) lov(U) osc(U) mdc(U) fid(U) fld(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) sha512_generic sha256_generic libcfs(U) autofs4 sunrpc ipv6 ppdev parport_pc parport microcode i2c_piix4 i2c_core snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc e1000 sg ext4 mbcache jbd2 sd_mod crc_t10dif sr_mod cdrom ahci pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib] Pid: 24218, comm: jbd2/dm-2-8 Not tainted 2.6.32.masterDEBUG11A #1 innotek GmbH VirtualBox RIP: 0010:[] [] kfree_debugcheck+0x30/0x40 RSP: 0018:ffff88002733dba0 EFLAGS: 00010082 RAX: 0000000000000039 RBX: 0006000100000002 RCX: 0000000000007a74 RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000046 RBP: ffff88002733dbb0 R08: 0000000000000000 R09: ffffffff8163acc0 R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000202 R13: 0006000100000002 R14: ffff880024d9d298 R15: ffff880024d9d298 FS: 0000000000000000(0000) GS:ffff880002200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: 0000003ac2ef5170 CR3: 000000003d0e0000 CR4: 00000000000006f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process jbd2/dm-2-8 (pid: 24218, threadinfo ffff88002733c000, task ffff88003d640ae0) Stack: ffff880000000020 ffffffffa035ebae ffff88002733dc00 ffffffff8116594b ffff88002851f720 ffff88003f810080 ffff88002733dc20 0006000100000002 ffff880024d9d240 0000000000000000 ffff880024d9d298 ffff880024d9d298 Call Trace: [] ? cfs_free+0xe/0x10 [libcfs] [] kfree+0x5b/0x2a0 [] cfs_free+0xe/0x10 [libcfs] [] lu_global_key_fini+0xa3/0xf0 [obdclass] [] key_fini+0x60/0x190 [obdclass] [] keys_fini+0x2f/0x120 [obdclass] [] lu_context_fini+0x2d/0xc0 [obdclass] [] osd_trans_commit_cb+0xe2/0x2b0 [osd_ldiskfs] [] ldiskfs_journal_commit_callback+0x8a/0xc0 [ldiskfs] [] jbd2_journal_commit_transaction+0x110f/0x1530 [jbd2] [] ? __switch_to+0xd0/0x320 [] ? try_to_del_timer_sync+0x7b/0xe0 [] kjournald2+0xb8/0x220 [jbd2] [] ? autoremove_wake_function+0x0/0x40 [] ? kjournald2+0x0/0x220 [jbd2] [] kthread+0x96/0xa0 [] child_rip+0xa/0x20 [] ? kthread+0x0/0xa0 [] ? child_rip+0x0/0x20 Code: 48 83 ec 08 0f 1f 44 00 00 48 89 fb e8 7a 67 ee ff 84 c0 74 07 48 83 c4 08 5b c9 c3 48 89 de 48 c7 c7 c8 0b 7a 81 e8 ed cc 39 00 <0f> 0b eb fe 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 RIP [] kfree_debugcheck+0x30/0x40 RSP ---[ end trace ff4011ce2a20c79c ]--- Kernel panic - not syncing: Fatal exception Pid: 24218, comm: jbd2/dm-2-8 Tainted: G D --------------- 2.6.32.masterDEBUG11A #1 Call Trace: [] ? panic+0xa0/0x168 [] ? oops_end+0xe4/0x100 [] ? die+0x5b/0x90 [] ? do_trap+0xc4/0x160 [] ? do_invalid_op+0x95/0xb0 [] ? kfree_debugcheck+0x30/0x40 [] ? libcfs_debug_vmsg2+0x4e3/0xb60 [libcfs] [] ? invalid_op+0x1b/0x20 [] ? kfree_debugcheck+0x30/0x40 [] ? cfs_free+0xe/0x10 [libcfs] [] ? kfree+0x5b/0x2a0 [] ? cfs_free+0xe/0x10 [libcfs] [] ? lu_global_key_fini+0xa3/0xf0 [obdclass] [] ? key_fini+0x60/0x190 [obdclass] [] ? keys_fini+0x2f/0x120 [obdclass] [] ? lu_context_fini+0x2d/0xc0 [obdclass] [] ? osd_trans_commit_cb+0xe2/0x2b0 [osd_ldiskfs] [] ? ldiskfs_journal_commit_callback+0x8a/0xc0 [ldiskfs] [] ? jbd2_journal_commit_transaction+0x110f/0x1530 [jbd2] [] ? __switch_to+0xd0/0x320 [] ? try_to_del_timer_sync+0x7b/0xe0 [] ? kjournald2+0xb8/0x220 [jbd2] [] ? autoremove_wake_function+0x0/0x40 [] ? kjournald2+0x0/0x220 [jbd2] [] ? kthread+0x96/0xa0 [] ? child_rip+0xa/0x20 [] ? kthread+0x0/0xa0 [] ? child_rip+0x0/0x20