
<ConMan> Console [onyx-138vm1] connected to <onyx-138:6000>.
[  783.353157] Autotest: Nodes provisioned in 25 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  783.782912] Autotest: Starting node configuration (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
         Stopping [0;1;39mSession 47 of User root[0m...
[[0;32m  OK  [0m] Removed slice [0;1;39mSlice /system/modprobe[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mGraphical Interface[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mMulti-User System[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mLogin Prompts[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mRPC Port Mapper[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mTimer Units[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDaily apt upgrade and clean activities[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDaily apt download activities[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDaily dpkg database backup timer[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mPeriodic ext4 Onli…ata Check for All Filesystems[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDiscard unused blocks once a week[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mRefresh fwupd metadata regularly[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDaily rotation of log files[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDaily man-db regeneration[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mMessage of the Day[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDaily Cleanup of Temporary Directories[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDownload data for …ailed at package install time[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mCheck to see wheth…w version of Ubuntu available[0m.
[[0;32m  OK  [0m] Closed [0;1;39mLVM2 poll daemon socket[0m.
[[0;32m  OK  [0m] Closed [0;1;39mLoad/Save RF Kill Switch Status /dev/rfkill Watch[0m.
         Stopping [0;1;39mModem Manager[0m...
         Stopping [0;1;39mLSB: automatic crash report generation[0m...
         Stopping [0;1;39mAutomounts filesystems on demand[0m...
         Stopping [0;1;39mAvailability of block devices[0m...
         Stopping [0;1;39mRegular background program processing daemon[0m...
         Stopping [0;1;39mCreate final runt…dir for shutdown pivot root[0m...
         Stopping [0;1;39mGetty on tty1[0m...
         Stopping [0;1;39mirqbalance daemon[0m...
         Stopping [0;1;39mKernel crash dump capture service[0m...
         Stopping [0;1;39mLSB: Load kernel image with kexec[0m...
         Stopping [0;1;39mMUNGE authentication service[0m...
         Stopping [0;1;39mDispatcher daemon for systemd-networkd[0m...
         Stopping [0;1;39mpNFS block layout mapping daemon[0m...
         Stopping [0;1;39mNFS server and services[0m...
         Stopping [0;1;39mPackageKit Daemon[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mTerminate Plymouth Boot Screen[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mPostfix Mail Transport Agent[0m.
         Stopping [0;1;39mPostfix Mail Transport Agent (instance -)[0m...
         Stopping [0;1;39mLSB: Start/Stop t…mote power manager) daemon.[0m...
         Stopping [0;1;39mSystem Logging Service[0m...
         Stopping [0;1;39mSerial Getty on ttyS0[0m...
         Stopping [0;1;39mSamba SMB Daemon[0m...
         Stopping [0;1;39mOpenBSD Secure Shell server[0m...
         Stopping [0;1;39mLoad/Save Random Seed[0m...
         Stopping [0;1;39mDisk Manager[0m...
         Stopping [0;1;39mUnattended Upgrades Shutdown[0m...
         Stopping [0;1;39mLSB: Start vmtouch daemon[0m...
         Stopping [0;1;39mLSB: Starts or stops the xinetd daemon.[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mpNFS block layout mapping daemon[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mirqbalance daemon[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDispatcher daemon for systemd-networkd[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDisk Manager[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mOpenBSD Secure Shell server[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mRegular background program processing daemon[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mSerial Getty on ttyS0[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mGetty on tty1[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mPackageKit Daemon[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mSession 47 of User root[0m.
[[0;32m  OK  [0m] Removed slice [0;1;39mSlice /system/getty[0m.
[[0;32m  OK  [0m] Removed slice [0;1;39mSlice /system/serial-getty[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mHold until boot process finishes up[0m.
         Stopping [0;1;39mUser Manager for UID 0[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mUser Manager for UID 0[0m.
         Stopping [0;1;39mUser Runtime Directory /run/user/0[0m...
[[0;32m  OK  [0m] Unmounted [0;1;39m/run/user/0[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mUser Runtime Directory /run/user/0[0m.
[[0;32m  OK  [0m] Removed slice [0;1;39mUser Slice of UID 0[0m.
         Stopping [0;1;39mPermit User Sessions[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mPermit User Sessions[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mAutomounts filesystems on demand[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mModem Manager[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mMUNGE authentication service[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mSamba SMB Daemon[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mUnattended Upgrades Shutdown[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mSystem Logging Service[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mPostfix Mail Transport Agent (instance -)[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mAvailability of block devices[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mLSB: Start/Stop th…remote power manager) daemon.[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mLSB: Starts or stops the xinetd daemon.[0m.
[  813.166573] nfsd: last server has exited, flushing export cache
[[0;32m  OK  [0m] Stopped [0;1;39mLSB: Start vmtouch daemon[0m.
[[0;32m  OK  [0m] Removed slice [0;1;39mSlice /system/postfix[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mSystem Time Synchronized[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mSystem Time Set[0m.
         Stopping [0;1;39mchrony, an NTP client/server[0m...
         Stopping [0;1;39mSamba NMB Daemon[0m...
         Stopping [0;1;39mAuthorization Manager[0m...
         Stopping [0;1;39mUser Login Management[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mLSB: automatic crash report generation[0m.
[  813.202684] kdump-tools[12101]: Stopping kdump-tools:
[  813.203601] kdump-tools[12201]:  * unloaded kdump kernel
[[0;32m  OK  [0m] Stopped [0;1;39mUser Login Management[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mKernel crash dump capture service[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mAuthorization Manager[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mSamba NMB Daemon[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mNFS server and services[0m.
         Stopping [0;1;39mNFSv4 ID-name mapping service[0m...
         Stopping [0;1;39mNFS Mount Daemon[0m...
         Stopping [0;1;39mNFSv4 Client Tracking Daemon[0m...
         Stopping [0;1;39mNFS status monitor for NFSv2/3 locking.[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mNFS status monitor for NFSv2/3 locking.[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mLSB: Load kernel image with kexec[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mHost and Network Name Lookups[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mRemote File Systems[0m.
         Unmounting [0;1;39m/autotest[0m...
         Unmounting [0;1;39m/scratch[0m...
         Stopping [0;1;39mLSB: Execute the …-e command to reboot system[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mchrony, an NTP client/server[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mNFSv4 ID-name mapping service[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mNFSv4 Client Tracking Daemon[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mNFS Mount Daemon[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mrpc_pipefs.target[0m.
         Unmounting [0;1;39mRPC Pipe File System[0m...
[[0;32m  OK  [0m] Unmounted [0;1;39mRPC Pipe File System[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mLSB: Execute the k…c -e command to reboot system[0m.
[[0;32m  OK  [0m] Unmounted [0;1;39m/autotest[0m.
[[0;32m  OK  [0m] Unmounted [0;1;39m/scratch[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mNetwork is Online[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mNetwork[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mPreparation for Remote File Systems[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mNFS client services[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mNetwork Manager Wait Online[0m.
         Stopping [0;1;39mNetwork Manager[0m...
         Stopping [0;1;39mNetwork Name Resolution[0m...
         Stopping [0;1;39mWPA supplicant[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mWPA supplicant[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mNetwork Name Resolution[0m.
         Stopping [0;1;39mNetwork Configuration[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mNetwork Configuration[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mNetwork Manager[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mBasic System[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mPreparation for Network[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mPath Units[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mSlice Units[0m.
[[0;32m  OK  [0m] Removed slice [0;1;39mUser and Session Slice[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mSocket Units[0m.
[[0;32m  OK  [0m] Closed [0;1;39mOpen-iSCSI iscsid Socket[0m.
[[0;32m  OK  [0m] Closed [0;1;39mSyslog Socket[0m.
[[0;32m  OK  [0m] Closed [0;1;39mNetwork Service Netlink Socket[0m.
[[0;32m  OK  [0m] Closed [0;1;39mUUID daemon activation socket[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mSystem Initialization[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mLocal Encrypted Volumes[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mForward Password R…uests to Wall Directory Watch[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mSwaps[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mLocal Verity Protected Volumes[0m.
         Deactivating swap [0;1;39m/swap.img[0m...
         Stopping [0;1;39mSet Up Additional Binary Formats[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mApply Kernel Variables[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mLoad Kernel Modules[0m.
         Stopping [0;1;39mRecord System Boot/Shutdown in UTMP[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mSet Up Additional Binary Formats[0m.
[[0;32m  OK  [0m] Unset automount [0;1;39mArbitrary …s File System Automount Point[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mRecord System Boot/Shutdown in UTMP[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mCreate Volatile Files and Directories[0m.
[[0;32m  OK  [0m] Deactivated swap [0;1;39m/swap.img[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mLoad/Save Random Seed[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mCreate final runtime dir for shutdown pivot root[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mLocal File Systems[0m.
         Unmounting [0;1;39m/run/credentials/systemd-sysusers.service[0m...
[[0;32m  OK  [0m] Unmounted [0;1;39m/run/credentials/systemd-sysusers.service[0m.
[[0;32m  OK  [0m] Stopped target [0;1;39mPreparation for Local File Systems[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mUnmount All Filesystems[0m.
         Stopping [0;1;39mMonitoring of LVM…meventd or progress polling[0m...
         Stopping [0;1;39mDevice-Mapper Multipath Device Controller[0m...
[[0;32m  OK  [0m] Stopped [0;1;39mCreate Static Device Nodes in /dev[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mCreate System Users[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mDevice-Mapper Multipath Device Controller[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mRemount Root and Kernel File Systems[0m.
[[0;32m  OK  [0m] Stopped [0;1;39mMonitoring of LVM2… dmeventd or progress polling[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mSystem Shutdown[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mLate Shutdown Services[0m.
[[0;32m  OK  [0m] Finished [0;1;39mSystem Reboot[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mSystem Reboot[0m.
[  814.380301] systemd-shutdown[1]: Syncing filesystems and block devices.
[  814.382587] systemd-journald[354]: Failed to send WATCHDOG=1 notification message: Connection refused
[  815.023274] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
[  815.030134] systemd-journald[354]: Received SIGTERM from PID 1 (systemd-shutdow).
[  815.719063] systemd-shutdown[1]: Sending SIGKILL to remaining processes...
[  815.723092] systemd-shutdown[1]: Unmounting file systems.
[  815.724944] [12750]: Remounting '/' read-only in with options '(null)'.
[  816.118627] EXT4-fs (vda1): re-mounted. Opts: (null). Quota mode: none.
[  816.121674] systemd-shutdown[1]: All filesystems unmounted.
[  816.122755] systemd-shutdown[1]: Deactivating swaps.
[  816.123754] systemd-shutdown[1]: All swaps deactivated.
[  816.124738] systemd-shutdown[1]: Detaching loop devices.
[  816.126668] systemd-shutdown[1]: All loop devices detached.
[  816.127722] systemd-shutdown[1]: Stopping MD devices.
[  816.128737] systemd-shutdown[1]: All MD devices stopped.
[  816.129733] systemd-shutdown[1]: Detaching DM devices.
[  816.130772] systemd-shutdown[1]: All DM devices detached.
[  816.131790] systemd-shutdown[1]: All filesystems, swaps, loop devices, MD devices and DM devices detached.
[  816.140710] systemd-shutdown[1]: Successfully changed into root pivot.
[  816.141959] systemd-shutdown[1]: Returning to initrd...
[  816.150595] shutdown[1]: Syncing filesystems and block devices.
[  816.152041] shutdown[1]: Sending SIGTERM to remaining processes...
[  816.154811] shutdown[1]: Sending SIGKILL to remaining processes...
[  816.157348] shutdown[1]: Unmounting file systems.
[  816.159022] sd-umoun[12756]: Unmounting '/oldroot/sys/kernel/config'.
[  816.160957] sd-umoun[12757]: Unmounting '/oldroot/sys/fs/fuse/connections'.
[  816.162939] sd-remou[12758]: Remounting '/oldroot/proc/fs/nfsd' read-only in with options '(null)'.
[  816.165163] sd-umoun[12759]: Unmounting '/oldroot/proc/fs/nfsd'.
[  816.167017] sd-umoun[12760]: Unmounting '/oldroot/sys/kernel/tracing'.
[  816.168832] sd-umoun[12761]: Unmounting '/oldroot/sys/kernel/debug'.
[  816.170609] sd-umoun[12762]: Unmounting '/oldroot/dev/mqueue'.
[  816.172311] sd-umoun[12763]: Unmounting '/oldroot/dev/hugepages'.
[  816.174048] sd-umoun[12764]: Unmounting '/oldroot/sys/fs/bpf'.
[  816.175739] sd-umoun[12765]: Unmounting '/oldroot/sys/fs/pstore'.
[  816.177471] sd-umoun[12766]: Unmounting '/oldroot/sys/fs/cgroup'.
[  816.179273] sd-umoun[12767]: Unmounting '/oldroot/run/lock'.
[  816.180958] sd-umoun[12768]: Unmounting '/oldroot/dev/shm'.
[  816.182744] sd-umoun[12769]: Unmounting '/oldroot/sys/kernel/security'.
[  816.184550] sd-umoun[12770]: Unmounting '/oldroot'.
[  816.185517] sd-umoun[12770]: Failed to unmount /oldroot: Device or resource busy
[  816.187436] sd-umoun[12771]: Unmounting '/oldroot/run'.
[  816.189026] sd-umoun[12772]: Unmounting '/oldroot/dev/pts'.
[  816.190757] sd-umoun[12773]: Unmounting '/oldroot/dev'.
[  816.192361] sd-umoun[12774]: Unmounting '/oldroot/proc'.
[  816.193970] sd-umoun[12775]: Unmounting '/oldroot/sys'.
[  816.195746] sd-umoun[12776]: Unmounting '/oldroot'.
[  816.403075] shutdown[1]: All filesystems unmounted.
[  816.404083] shutdown[1]: Deactivating swaps.
[  816.404951] shutdown[1]: All swaps deactivated.
[  816.405844] shutdown[1]: Detaching loop devices.
[  816.407673] shutdown[1]: All loop devices detached.
[  816.408632] shutdown[1]: Stopping MD devices.
[  816.409555] shutdown[1]: All MD devices stopped.
[  816.410485] shutdown[1]: Detaching DM devices.
[  816.411417] shutdown[1]: All DM devices detached.
[  816.412337] shutdown[1]: All filesystems, swaps, loop devices, MD devices and DM devices detached.
[  816.420787] shutdown[1]: Syncing filesystems and block devices.
[  816.421967] shutdown[1]: Rebooting.
[  816.423663] reboot: Restarting system
[  816.424374] reboot: machine restart
[1;256r[256;256H[6n
Google, Inc.
Serial Graphics Adapter 06/09/14
SGABIOS $Id: sgabios.S 8 2010-04-22 00:03:40Z nlaredo $ (mockbuild@) Mon Jun  9 21:33:48 UTC 2014
Term: 80x24
4 0
[2JSeaBIOS (version 1.11.0-2.el7)
Machine UUID c6bd1012-4322-4821-9185-d64b65c0d53e


iPXE (http://ipxe.org) 00:03.0 C100 PCI2.10 PnP PMM+BFF945D0+BFED45D0 C100
Press Ctrl-B to configure iPXE (PCI 00:03.0)...                                                                               


Booting from ROM...
iPXE (PCI 00:03.0) starting execution...ok
iPXE initialising devices...ok



[40;37;1mi[mi[40;37;1mP[mP[40;37;1mX[mX[40;37;1mE[mE[40;37;1m [m [40;37;1m1[m1[40;37;1m.[m.[40;37;1m0[m0[40;37;1m.[m.[40;37;1m0[m0[40;37;1m+[m+[40;37;1m [m [40;37;1m([m([40;37;1m1[m1[40;37;1m3[m3[40;37;1m3[m3[40;37;1mf[mf[40;37;1m4[m4[40;37;1mc[mc[40;37;1m)[m) -- Open Source Network Boot Firmware -- [40;36;22mh[mh[40;36;22mt[mt[40;36;22mt[mt[40;36;22mp[mp[40;36;22m:[m:[40;36;22m/[m/[40;36;22m/[m/[40;36;22mi[mi[40;36;22mp[mp[40;36;22mx[mx[40;36;22me[me[40;36;22m.[m.[40;36;22mo[mo[40;36;22mr[mr[40;36;22mg[mg
Features: DNS HTTP iSCSI TFTP VLAN AoE ELF MBOOT PXE bzImage Menu PXEXT

Press Ctrl-B for the iPXE command line...                                         net0: 52:54:00:90:ae:8f using virtio-net on 0000:00:03.0 (open)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
Configuring (net0 52:54:00:90:ae:8f).............. ok
net0: 10.240.22.180/255.255.240.0 gw 10.240.16.1
Next server: 10.240.16.5
Filename: /pxelinux.0
tftp://10.240.16.5//pxelinux.0... ok
pxelinux.0 : 26759 bytes [PXE-NBP]
[1SPXELINUX 4.05 0x5bd8f633  Copyright (C) 1994-2011 H. Peter Anvin et al[1S!PXE entry point found (we hope) at 9C2C:0160 via plan A[1SUNDI code segment at 9C2C len 0802[1SUNDI data segment at 9CB0 len 2CE0[1SGetting cached packet  01 02 03[1SMy IP address seems to be 0AF016B4 10.240.22.180[1Sip=10.240.22.180:10.240.16.5:10.240.16.1:255.255.240.0[1SBOOTIF=01-52-54-00-90-ae-8f[1SSYSUUID=1210bdc6-2243-2148-9185-d64b65c0d53e[1STFTP prefix: /[1STrying to load: pxelinux.cfg/1210bdc6-2243-2148-9185-d64b65c0d53e   Trying to load: pxelinux.cfg/01-52-54-00-90-ae-8f                   ok[1SBooting from local disk...[1S
No more network devices

Press Ctrl-B for the iPXE command line...                                         Booting from Hard Disk...
Loading Linux 5.15.0-94-generic ...
Loading initial ramdisk ...
[    0.000000] Linux version 5.15.0-94-generic (buildd@lcy02-amd64-096) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 (Ubuntu 5.15.0-94.104-generic 5.15.136)
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=55af9729-2f39-4ca5-a7b8-5098bfdaf6ee ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 slab_nomerge slub_nomerge crashkernel=512M-:296M
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Hygon HygonGenuine
[    0.000000]   Centaur CentaurHauls
[    0.000000]   zhaoxin   Shanghai  
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfff9fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000bfffa000-0x00000000bfffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.4 present.
[    0.000000] DMI: Red Hat KVM, BIOS 0.5.1 01/01/2011
[    0.000000] Hypervisor detected: KVM
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 1b201001, primary cpu clock
[    0.000004] kvm-clock: using sched offset of 858317280196 cycles
[    0.000010] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.000018] tsc: Detected 2099.998 MHz processor
[    0.000305] last_pfn = 0x140000 max_arch_pfn = 0x400000000
[    0.000359] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[    0.000379] last_pfn = 0xbfffa max_arch_pfn = 0x400000000
[    0.011466] found SMP MP-table at [mem 0x000f6390-0x000f639f]
[    0.011499] Using GB pages for direct mapping
[    0.011807] RAMDISK: [mem 0x2b847000-0x31c1afff]
[    0.011827] ACPI: Early table checksum verification disabled
[    0.011842] ACPI: RSDP 0x00000000000F61A0 000014 (v00 BOCHS )
[    0.011853] ACPI: RSDT 0x00000000BFFFFB40 000030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.011870] ACPI: FACP 0x00000000BFFFF177 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.011879] ACPI: DSDT 0x00000000BFFFE040 001137 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.011888] ACPI: FACS 0x00000000BFFFE000 000040
[    0.011893] ACPI: SSDT 0x00000000BFFFF1EB 0008D5 (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
[    0.011898] ACPI: APIC 0x00000000BFFFFAC0 000080 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.011902] ACPI: Reserving FACP table memory at [mem 0xbffff177-0xbffff1ea]
[    0.011905] ACPI: Reserving DSDT table memory at [mem 0xbfffe040-0xbffff176]
[    0.011906] ACPI: Reserving FACS table memory at [mem 0xbfffe000-0xbfffe03f]
[    0.011908] ACPI: Reserving SSDT table memory at [mem 0xbffff1eb-0xbffffabf]
[    0.011909] ACPI: Reserving APIC table memory at [mem 0xbffffac0-0xbffffb3f]
[    0.012392] No NUMA configuration found
[    0.012395] Faking a node at [mem 0x0000000000000000-0x000000013fffffff]
[    0.012414] NODE_DATA(0) allocated [mem 0x13ffd4000-0x13fffdfff]
[    0.012841] Reserving 296MB of memory at 2768MB for crashkernel (System RAM: 4095MB)
[    0.012872] Zone ranges:
[    0.012873]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.012876]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.012878]   Normal   [mem 0x0000000100000000-0x000000013fffffff]
[    0.012880]   Device   empty
[    0.012882] Movable zone start for each node
[    0.012886] Early memory node ranges
[    0.012887]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.012889]   node   0: [mem 0x0000000000100000-0x00000000bfff9fff]
[    0.012891]   node   0: [mem 0x0000000100000000-0x000000013fffffff]
[    0.012893] Initmem setup node 0 [mem 0x0000000000001000-0x000000013fffffff]
[    0.012907] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.013001] On node 0, zone DMA: 97 pages in unavailable ranges
[    0.027955] On node 0, zone Normal: 6 pages in unavailable ranges
[    0.028531] ACPI: PM-Timer IO Port: 0x608
[    0.028556] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.028605] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.028610] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.028613] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.028615] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.028621] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.028623] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.028629] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.028632] TSC deadline timer available
[    0.028634] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[    0.028672] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.028676] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.028677] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.028679] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.028681] PM: hibernation: Registered nosave memory: [mem 0xbfffa000-0xbfffffff]
[    0.028683] PM: hibernation: Registered nosave memory: [mem 0xc0000000-0xfeffbfff]
[    0.028685] PM: hibernation: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
[    0.028687] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xfffbffff]
[    0.028688] PM: hibernation: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
[    0.028691] [mem 0xc0000000-0xfeffbfff] available for PCI devices
[    0.028693] Booting paravirtualized kernel on KVM
[    0.028705] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.028719] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[    0.029202] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u1048576
[    0.029244] kvm-guest: stealtime: cpu 0, msr 13bc33080
[    0.029249] kvm-guest: PV spinlocks enabled
[    0.029252] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.029271] Built 1 zonelists, mobility grouping on.  Total pages: 1031930
[    0.029274] Policy zone: Normal
[    0.029275] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic root=UUID=55af9729-2f39-4ca5-a7b8-5098bfdaf6ee ro console=tty0 console=ttyS0,115200 biosdevname=0 crashkernel=320M net.ifnames=0 console=tty0 console=ttyS0,115200 slab_nomerge slub_nomerge crashkernel=512M-:296M
[    0.029463] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.0-94-generic biosdevname=0", will be passed to user space.
[    0.030183] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.030547] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.030618] mem auto-init: stack:off, heap alloc:on, heap free:off
[    0.062273] Memory: 3591544K/4193888K available (16393K kernel code, 4394K rwdata, 10876K rodata, 3356K init, 18716K bss, 602084K reserved, 0K cma-reserved)
[    0.062645] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.062673] Kernel/User page tables isolation: enabled
[    0.062728] ftrace: allocating 50666 entries in 198 pages
[    0.094254] ftrace: allocated 198 pages with 4 groups
[    0.094446] rcu: Hierarchical RCU implementation.
[    0.094449] rcu: 	RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=2.
[    0.094451] 	Rude variant of Tasks RCU enabled.
[    0.094452] 	Tracing variant of Tasks RCU enabled.
[    0.094456] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.094458] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.099294] NR_IRQS: 524544, nr_irqs: 440, preallocated irqs: 16
[    0.099500] random: crng init done
[    0.100488] Console: colour VGA+ 80x25
[    0.112692] printk: console [tty0] enabled
[    0.238975] printk: console [ttyS0] enabled
[    0.239827] ACPI: Core revision 20210730
[    0.240727] APIC: Switch to symmetric I/O mode setup
[    0.242006] x2apic enabled
[    0.242929] Switched APIC routing to physical x2apic.
[    0.245281] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.246586] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x1e452ea631d, max_idle_ns: 440795244572 ns
[    0.248737] Calibrating delay loop (skipped) preset value.. 4199.99 BogoMIPS (lpj=8399992)
[    0.250601] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.251741] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.252739] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.254487] Spectre V2 : Mitigation: Retpolines
[    0.255471] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.256734] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[    0.258117] Spectre V2 : Enabling Restricted Speculation for firmware calls
[    0.259545] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.260735] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.262663] MDS: Mitigation: Clear CPU buffers
[    0.264733] TAA: Mitigation: Clear CPU buffers
[    0.265696] MMIO Stale Data: Unknown: No mitigations
[    0.266744] SRBDS: Unknown: Dependent on hypervisor status
[    0.267924] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.268733] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.270053] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.271369] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.272734] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[    0.298047] Freeing SMP alternatives memory: 44K
[    0.299088] pid_max: default: 32768 minimum: 301
[    0.300104] LSM: Security Framework initializing
[    0.300747] landlock: Up and running.
[    0.301561] Yama: becoming mindful.
[    0.302381] AppArmor: AppArmor initialized
[    0.303319] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.304748] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.306703] smpboot: CPU0: Intel Core Processor (Broadwell, IBRS) (family: 0x6, model: 0x3d, stepping: 0x2)
[    0.308732] Performance Events: unsupported p6 CPU model 61 no PMU driver, software events only.
[    0.308732] signal: max sigframe size: 1776
[    0.308768] rcu: Hierarchical SRCU implementation.
[    0.310283] NMI watchdog: Perf NMI watchdog permanently disabled
[    0.311604] smp: Bringing up secondary CPUs ...
[    0.312729] x86: Booting SMP configuration:
[    0.312739] .... node  #0, CPUs:      #1
[    0.145241] kvm-clock: cpu 1, msr 1b201041, secondary cpu clock
[    0.145241] smpboot: CPU 1 Converting physical 0 to logical die 1
[    0.316767] kvm-guest: stealtime: cpu 1, msr 13bd33080
[    0.317781] smp: Brought up 1 node, 2 CPUs
[    0.317781] smpboot: Max logical packages: 2
[    0.318427] smpboot: Total of 2 processors activated (8399.99 BogoMIPS)
[    0.321171] devtmpfs: initialized
[    0.321475] x86/mm: Memory block size: 128MB
[    0.322777] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.324748] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[    0.326058] pinctrl core: initialized pinctrl subsystem
[    0.327288] PM: RTC time: 23:31:04, date: 2024-12-16
[    0.328449] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.329000] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.330767] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.332324] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.332749] audit: initializing netlink subsys (disabled)
[    0.333853] audit: type=2000 audit(1734391802.120:1): state=initialized audit_enabled=0 res=1
[    0.333853] thermal_sys: Registered thermal governor 'fair_share'
[    0.334319] thermal_sys: Registered thermal governor 'bang_bang'
[    0.335485] thermal_sys: Registered thermal governor 'step_wise'
[    0.336737] thermal_sys: Registered thermal governor 'user_space'
[    0.337867] thermal_sys: Registered thermal governor 'power_allocator'
[    0.339011] EISA bus registered
[    0.340745] cpuidle: using governor ladder
[    0.341555] cpuidle: using governor menu
[    0.342455] ACPI: bus type PCI registered
[    0.343261] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.344641] PCI: Using configuration type 1 for base access
[    0.346271] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.347942] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.348742] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.350045] ACPI: Added _OSI(Module Device)
[    0.352738] ACPI: Added _OSI(Processor Device)
[    0.353611] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.354531] ACPI: Added _OSI(Processor Aggregator Device)
[    0.355558] ACPI: Added _OSI(Linux-Dell-Video)
[    0.356421] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.356744] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.359363] ACPI: 2 ACPI AML tables successfully acquired and loaded
[    0.361526] ACPI: Interpreter enabled
[    0.362311] ACPI: PM: (supports S0 S5)
[    0.363067] ACPI: Using IOAPIC for interrupt routing
[    0.364041] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.364736] PCI: Using E820 reservations for host bridge windows
[    0.366139] ACPI: Enabled 16 GPEs in block 00 to 0F
[    0.370076] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.371287] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[    0.372750] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[    0.375183] acpiphp: Slot [3] registered
[    0.376016] acpiphp: Slot [4] registered
[    0.376770] acpiphp: Slot [5] registered
[    0.377582] acpiphp: Slot [6] registered
[    0.378389] acpiphp: Slot [7] registered
[    0.379197] acpiphp: Slot [8] registered
[    0.379996] acpiphp: Slot [9] registered
[    0.380760] acpiphp: Slot [10] registered
[    0.381584] acpiphp: Slot [11] registered
[    0.382413] acpiphp: Slot [12] registered
[    0.383234] acpiphp: Slot [13] registered
[    0.384054] acpiphp: Slot [14] registered
[    0.384759] acpiphp: Slot [15] registered
[    0.385587] acpiphp: Slot [16] registered
[    0.386416] acpiphp: Slot [17] registered
[    0.387236] acpiphp: Slot [18] registered
[    0.388061] acpiphp: Slot [19] registered
[    0.388759] acpiphp: Slot [20] registered
[    0.389579] acpiphp: Slot [21] registered
[    0.390401] acpiphp: Slot [22] registered
[    0.391225] acpiphp: Slot [23] registered
[    0.392048] acpiphp: Slot [24] registered
[    0.392765] acpiphp: Slot [25] registered
[    0.393590] acpiphp: Slot [26] registered
[    0.394439] acpiphp: Slot [27] registered
[    0.395268] acpiphp: Slot [28] registered
[    0.396093] acpiphp: Slot [29] registered
[    0.396762] acpiphp: Slot [30] registered
[    0.397577] acpiphp: Slot [31] registered
[    0.398396] PCI host bridge to bus 0000:00
[    0.399219] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.400271] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.400736] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.402001] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.403393] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]
[    0.404842] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[    0.406943] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[    0.408928] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[    0.416661] pci 0000:00:01.1: reg 0x20: [io  0xc0a0-0xc0af]
[    0.419359] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
[    0.420699] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
[    0.420737] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
[    0.422060] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
[    0.423566] pci 0000:00:01.2: [8086:7020] type 00 class 0x0c0300
[    0.430695] pci 0000:00:01.2: reg 0x20: [io  0xc040-0xc05f]
[    0.434395] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[    0.436154] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
[    0.436750] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
[    0.439050] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
[    0.442011] pci 0000:00:03.0: reg 0x10: [io  0xc060-0xc07f]
[    0.444737] pci 0000:00:03.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff]
[    0.456502] pci 0000:00:03.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref]
[    0.457633] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000
[    0.460737] pci 0000:00:04.0: reg 0x10: [io  0xc000-0xc03f]
[    0.464339] pci 0000:00:04.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff]
[    0.476082] pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00
[    0.478146] pci 0000:00:05.0: reg 0x10: [io  0xc080-0xc09f]
[    0.493967] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[    0.495258] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[    0.496537] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
[    0.496870] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[    0.498072] ACPI: PCI: Interrupt link LNKS configured for IRQ 9
[    0.499515] iommu: Default domain type: Translated 
[    0.499515] iommu: DMA domain TLB invalidation policy: lazy mode 
[    0.500924] SCSI subsystem initialized
[    0.501721] vgaarb: loaded
[    0.501721] ACPI: bus type USB registered
[    0.502183] usbcore: registered new interface driver usbfs
[    0.503230] usbcore: registered new interface driver hub
[    0.504744] usbcore: registered new device driver usb
[    0.505734] pps_core: LinuxPPS API ver. 1 registered
[    0.506684] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.508330] PTP clock support registered
[    0.508779] EDAC MC: Ver: 3.0.0
[    0.509644] NetLabel: Initializing
[    0.509644] NetLabel:  domain hash size = 128
[    0.510457] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.511546] NetLabel:  unlabeled traffic allowed by default
[    0.512759] PCI: Using ACPI for IRQ routing
[    0.514248] clocksource: Switched to clocksource kvm-clock
[    0.531831] VFS: Disk quotas dquot_6.6.0
[    0.532654] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.534054] AppArmor: AppArmor Filesystem Enabled
[    0.534988] pnp: PnP ACPI init
[    0.536159] pnp: PnP ACPI: found 5 devices
[    0.543775] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.545469] NET: Registered PF_INET protocol family
[    0.546547] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.549148] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.550808] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.552314] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.553927] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.555317] TCP: Hash tables configured (established 32768 bind 32768)
[    0.556625] MPTCP token hash table entries: 4096 (order: 4, 98304 bytes, linear)
[    0.558056] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.559325] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.560670] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.561765] NET: Registered PF_XDP protocol family
[    0.562699] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.563857] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.565014] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    0.566290] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window]
[    0.567601] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    0.568718] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    0.569855] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    0.571374] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.573126] PCI: CLS 0 bytes, default 64
[    0.573998] Trying to unpack rootfs image as initramfs...
[    0.581567] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.582816] software IO TLB: mapped [mem 0x00000000a9000000-0x00000000ad000000] (64MB)
[    0.584389] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1e452ea631d, max_idle_ns: 440795244572 ns
[    0.593549] Initialise system trusted keyrings
[    0.594466] Key type blacklist registered
[    0.601295] workingset: timestamp_bits=36 max_order=20 bucket_order=0
[    0.604097] zbud: loaded
[    0.605084] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.606632] fuse: init (API version 7.34)
[    0.607808] integrity: Platform Keyring initialized
[    0.621293] Key type asymmetric registered
[    0.622135] Asymmetric key parser 'x509' registered
[    0.623116] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
[    0.625284] io scheduler mq-deadline registered
[    0.626467] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.627865] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    0.629287] ACPI: button: Power Button [PWRF]
[    0.633714] ACPI: \_SB_.LNKC: Enabled at IRQ 10
[    0.634689] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
[    0.636278] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[    0.638526] ACPI: \_SB_.LNKA: Enabled at IRQ 10
[    0.639459] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[    0.641782] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    0.666173] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.669189] Linux agpgart interface v0.103
[    0.671640] loop: module loaded
[    0.673846] scsi host0: ata_piix
[    0.674660] scsi host1: ata_piix
[    0.675376] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0a0 irq 14
[    0.676632] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0a8 irq 15
[    0.678027] tun: Universal TUN/TAP device driver, 1.6
[    0.679321] PPP generic driver version 2.4.2
[    0.680216] VFIO - User Level meta-driver version: 0.3
[    0.681577] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.682807] ehci-pci: EHCI PCI platform driver
[    0.683677] ehci-platform: EHCI generic platform driver
[    0.684684] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.685861] ohci-pci: OHCI PCI platform driver
[    0.686737] ohci-platform: OHCI generic platform driver
[    0.687747] uhci_hcd: USB Universal Host Controller Interface driver
[    0.690126] uhci_hcd 0000:00:01.2: UHCI Host Controller
[    0.691153] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
[    0.692535] uhci_hcd 0000:00:01.2: detected 2 ports
[    0.693621] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c040
[    0.694812] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 5.15
[    0.696368] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.697750] usb usb1: Product: UHCI Host Controller
[    0.698712] usb usb1: Manufacturer: Linux 5.15.0-94-generic uhci_hcd
[    0.699951] usb usb1: SerialNumber: 0000:00:01.2
[    0.700931] hub 1-0:1.0: USB hub found
[    0.701714] hub 1-0:1.0: 2 ports detected
[    0.702772] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    0.705239] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.706261] serio: i8042 AUX port at 0x60,0x64 irq 12
[    0.707367] mousedev: PS/2 mouse device common for all mice
[    0.708585] rtc_cmos 00:00: RTC can wake from S4
[    0.710395] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    0.712291] rtc_cmos 00:00: registered as rtc0
[    0.713345] rtc_cmos 00:00: setting system clock to 2024-12-16T23:31:04 UTC (1734391864)
[    0.714937] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
[    0.716151] i2c_dev: i2c /dev entries driver
[    0.717011] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[    0.719296] device-mapper: uevent: version 1.0.3
[    0.720293] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[    0.721906] platform eisa.0: Probing EISA bus 0
[    0.722824] platform eisa.0: EISA: Cannot allocate resource for mainboard
[    0.724111] platform eisa.0: Cannot allocate resource for EISA slot 1
[    0.725370] platform eisa.0: Cannot allocate resource for EISA slot 2
[    0.726597] platform eisa.0: Cannot allocate resource for EISA slot 3
[    0.727822] platform eisa.0: Cannot allocate resource for EISA slot 4
[    0.729046] platform eisa.0: Cannot allocate resource for EISA slot 5
[    0.730301] platform eisa.0: Cannot allocate resource for EISA slot 6
[    0.731534] platform eisa.0: Cannot allocate resource for EISA slot 7
[    0.732770] platform eisa.0: Cannot allocate resource for EISA slot 8
[    0.734000] platform eisa.0: EISA: Detected 0 cards
[    0.734971] intel_pstate: CPU model not supported
[    0.735962] ledtrig-cpu: registered to indicate activity on CPUs
[    0.737202] drop_monitor: Initializing network drop monitor service
[    0.738634] NET: Registered PF_INET6 protocol family
[    1.310964] Freeing initrd memory: 102224K
[    1.319138] Segment Routing with IPv6
[    1.319914] In-situ OAM (IOAM) with IPv6
[    1.320712] NET: Registered PF_PACKET protocol family
[    1.321750] Key type dns_resolver registered
[    1.322861] IPI shorthand broadcast: enabled
[    1.323716] sched_clock: Marking stable (1181490237, 141241233)->(1374359308, -51627838)
[    1.325344] registered taskstats version 1
[    1.326238] Loading compiled-in X.509 certificates
[    1.327900] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[    1.330325] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969'
[    1.332684] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19'
[    1.334477] blacklist: Loading compiled-in revocation X.509 certificates
[    1.335726] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing: 61482aa2830d0ab2ad5af10b7250da9033ddcef0'
[    1.337518] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2017): 242ade75ac4a15e50d50c84b0d45ff3eae707a03'
[    1.339385] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (ESM 2018): 365188c1d374d6b07c3c8f240f8ef722433d6a8b'
[    1.341317] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2019): c0746fd6c5da3ae827864651ad66ae47fe24b3e8'
[    1.343189] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d'
[    1.345100] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v2): 4cf046892d6fd3c9a5b03f98d845f90851dc6a8c'
[    1.347015] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v3): 100437bb6de6e469b581e61cd66bce3ef4ed53af'
[    1.348928] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9'
[    1.351124] zswap: loaded using pool lzo/zbud
[    1.352150] Key type .fscrypt registered
[    1.352937] Key type fscrypt-provisioning registered
[    1.356777] Key type encrypted registered
[    1.357588] AppArmor: AppArmor sha1 policy hashing enabled
[    1.358625] ima: No TPM chip found, activating TPM-bypass!
[    1.359658] Loading compiled-in module X.509 certificates
[    1.361307] Loaded X.509 cert 'Build time autogenerated kernel key: 32d1800ed0f0bcec93454c39c50cea5381b35496'
[    1.363076] ima: Allocated hash algorithm: sha1
[    1.363953] ima: No architecture policies found
[    1.364837] evm: Initialising EVM extended attributes:
[    1.365815] evm: security.selinux
[    1.366483] evm: security.SMACK64
[    1.367154] evm: security.SMACK64EXEC
[    1.367881] evm: security.SMACK64TRANSMUTE
[    1.368676] evm: security.SMACK64MMAP
[    1.369408] evm: security.apparmor
[    1.370095] evm: security.ima
[    1.370706] evm: security.capability
[    1.371418] evm: HMAC attrs: 0x1
[    1.372331] PM:   Magic number: 4:160:555
[    1.373357] RAS: Correctable Errors collector initialized.
[    1.376190] Freeing unused decrypted memory: 2036K
[    1.378454] Freeing unused kernel image (initmem) memory: 3356K
[    1.397258] Write protecting the kernel read-only data: 30720k
[    1.400333] Freeing unused kernel image (text/rodata gap) memory: 2036K
[    1.402027] Freeing unused kernel image (rodata/data gap) memory: 1412K
[    1.456267] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    1.457489] x86/mm: Checking user space page tables
[    1.509769] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    1.511034] Run /init as init process
Loading, please wait...
Starting version 249.11-0ubuntu3.12
[    1.645798] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
[    1.646360] virtio_blk virtio1: [vda] 209715200 512-byte logical blocks (107 GB/100 GiB)
[    1.647902] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
[    1.652005]  vda: vda1 vda4
[    1.652081] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
[    1.657843] cryptd: max_cpu_qlen set to 1000
[    1.663884] AVX2 version of gcm_enc/dec engaged.
[    1.664939] AES CTR mode by8 optimization enabled
[    1.666805] FDC 0 is a S82078B
Begin: Loading essential drivers ... [    2.937262] raid6: avx2x4   gen() 25128 MB/s
[    3.005251] raid6: avx2x4   xor()  8851 MB/s
[    3.073261] raid6: avx2x2   gen() 23796 MB/s
[    3.141251] raid6: avx2x2   xor() 14967 MB/s
[    3.209259] raid6: avx2x1   gen() 22327 MB/s
[    3.277250] raid6: avx2x1   xor() 13117 MB/s
[    3.345249] raid6: sse2x4   gen() 13534 MB/s
[    3.413262] raid6: sse2x4   xor()  7532 MB/s
[    3.481260] raid6: sse2x2   gen() 12919 MB/s
[    3.549250] raid6: sse2x2   xor()  8590 MB/s
[    3.617250] raid6: sse2x1   gen() 11377 MB/s
[    3.685250] raid6: sse2x1   xor()  7537 MB/s
[    3.686115] raid6: using algorithm avx2x4 gen() 25128 MB/s
[    3.687169] raid6: .... xor() 8851 MB/s, rmw enabled
[    3.688133] raid6: using avx2x2 recovery algorithm
[    3.690124] xor: automatically using best checksumming function   avx       
[    3.692343] async_tx: api initialized (async)
done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... [    3.764613] Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes
Scanning for Btrfs filesystems
done.
Begin: Will now check root file system ... fsck from util-linux 2.37.2
[/usr/sbin/fsck.ext4 (1) -- /dev/vda1] fsck.ext4 -a -C0 /dev/vda1 
/dev/vda1: clean, 151591/1310720 files, 2472599/5242880 blocks
done.
[    3.971518] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[    4.121839] systemd[1]: Inserted module 'autofs4'
[    4.134398] systemd[1]: systemd 249.11-0ubuntu3.12 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    4.139888] systemd[1]: Detected virtualization kvm.
[    4.140890] systemd[1]: Detected architecture x86-64.

Welcome to [1mUbuntu 22.04.5 LTS[0m!

[    4.143161] systemd[1]: Hostname set to <onyx-138vm1.onyx.whamcloud.com>.
[    4.247557] systemd[1]: Configuration file /run/systemd/system/netplan-ovs-cleanup.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
[    4.311373] systemd[1]: Queued start job for default target Graphical Interface.
[    4.313959] systemd[1]: Created slice Slice /system/modprobe.
[[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/modprobe[0m.
[    4.316326] systemd[1]: Created slice Slice /system/postfix.
[[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/postfix[0m.
[    4.318590] systemd[1]: Created slice Slice /system/serial-getty.
[[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/serial-getty[0m.
[    4.320819] systemd[1]: Created slice User and Session Slice.
[[0;32m  OK  [0m] Created slice [0;1;39mUser and Session Slice[0m.
[    4.322810] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[[0;32m  OK  [0m] Started [0;1;39mForward Password R…uests to Wall Directory Watch[0m.
[    4.325482] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[[0;32m  OK  [0m] Set up automount [0;1;39mArbitrary…s File System Automount Point[0m.
[    4.328287] systemd[1]: Reached target Slice Units.
[[0;32m  OK  [0m] Reached target [0;1;39mSlice Units[0m.
[    4.330001] systemd[1]: Reached target System Time Set.
[[0;32m  OK  [0m] Reached target [0;1;39mSystem Time Set[0m.
[    4.331820] systemd[1]: Reached target Local Verity Protected Volumes.
[[0;32m  OK  [0m] Reached target [0;1;39mLocal Verity Protected Volumes[0m.
[    4.334027] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[[0;32m  OK  [0m] Listening on [0;1;39mDevice-mapper event daemon FIFOs[0m.
[    4.336299] systemd[1]: Listening on LVM2 poll daemon socket.
[[0;32m  OK  [0m] Listening on [0;1;39mLVM2 poll daemon socket[0m.
[    4.338369] systemd[1]: Listening on multipathd control socket.
[[0;32m  OK  [0m] Listening on [0;1;39mmultipathd control socket[0m.
[    4.344049] systemd[1]: Listening on RPCbind Server Activation Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mRPCbind Server Activation Socket[0m.
[    4.346434] systemd[1]: Listening on Syslog Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mSyslog Socket[0m.
[    4.348222] systemd[1]: Listening on fsck to fsckd communication Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mfsck to fsckd communication Socket[0m.
[    4.350470] systemd[1]: Listening on initctl Compatibility Named Pipe.
[[0;32m  OK  [0m] Listening on [0;1;39minitctl Compatibility Named Pipe[0m.
[    4.352823] systemd[1]: Listening on Journal Audit Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mJournal Audit Socket[0m.
[    4.354784] systemd[1]: Listening on Journal Socket (/dev/log).
[[0;32m  OK  [0m] Listening on [0;1;39mJournal Socket (/dev/log)[0m.
[    4.356879] systemd[1]: Listening on Journal Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mJournal Socket[0m.
[    4.358743] systemd[1]: Listening on Network Service Netlink Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mNetwork Service Netlink Socket[0m.
[    4.360934] systemd[1]: Listening on udev Control Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mudev Control Socket[0m.
[    4.362873] systemd[1]: Listening on udev Kernel Socket.
[[0;32m  OK  [0m] Listening on [0;1;39mudev Kernel Socket[0m.
[    4.365587] systemd[1]: Mounting Huge Pages File System...
         Mounting [0;1;39mHuge Pages File System[0m...
[    4.368323] systemd[1]: Mounting POSIX Message Queue File System...
         Mounting [0;1;39mPOSIX Message Queue File System[0m...
[    4.371124] systemd[1]: Mounting NFSD configuration filesystem...
         Mounting [0;1;39mNFSD configuration filesystem[0m...
[    4.374119] systemd[1]: Mounting Kernel Debug File System...
         Mounting [0;1;39mKernel Debug File System[0m...
[    4.376946] systemd[1]: Mounting Kernel Trace File System...
         Mounting [0;1;39mKernel Trace File System[0m...
[    4.380594] systemd[1]: Starting Journal Service...
         Starting [0;1;39mJournal Service[0m...
[    4.382432] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS being skipped.
[    4.385165] systemd[1]: Starting Set the console keyboard layout...
         Starting [0;1;39mSet the console keyboard layout[0m...
[    4.397790] systemd[1]: Starting Create List of Static Device Nodes...
         Starting [0;1;39mCreate List of Static Device Nodes[0m...
[    4.407026] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
         Starting [0;1;39mMonitoring of LVM…meventd or progress polling[0m...
[    4.415324] systemd[1]: Condition check resulted in LXD - agent being skipped.
[    4.421371] RPC: Registered named UNIX socket transport module.
[    4.421740] systemd[1]: Starting Load Kernel Module configfs...
[    4.422587] RPC: Registered udp transport module.
[    4.424648] RPC: Registered tcp transport module.
[    4.424649] RPC: Registered tcp NFSv4.1 backchannel transport module.
         Starting [0;1;39mLoad Kernel Module configfs[0m...
[    4.441463] systemd[1]: Starting Load Kernel Module drm...
         Starting [0;1;39mLoad Kernel Module drm[0m...
[    4.452617] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[    4.458860] systemd[1]: Starting Load Kernel Module efi_pstore...
         Starting [0;1;39mLoad Kernel Module efi_pstore[0m...
[    4.470879] systemd[1]: Starting Load Kernel Module fuse...
         Starting [0;1;39mLoad Kernel Module fuse[0m...
[    4.472741] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
[    4.474817] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[    4.477929] systemd[1]: Starting Load Kernel Modules...
         Starting [0;1;39mLoad Kernel Modules[0m...
[    4.480766] systemd[1]: Starting Remount Root and Kernel File Systems...
         Starting [0;1;39mRemount Root and Kernel File Systems[0m...
[    4.489218] systemd[1]: Starting Coldplug All udev Devices...
         Starting [0;1;39mColdplug All udev Devices[0m...
[    4.493200] systemd[1]: Started Journal Service.
[[0;32m  OK  [0m] Started [0;1;39mJournal Service[0m.
[[0;32m  OK  [0m] Mounted [0;1;39mHuge Pages File System[0m.
[[0;32m  OK  [0m] Mounted [0;1;39mPOSIX Message Queue File System[0m.
[[0;32m  OK  [0m] Mounted [0;1;39mNFSD configuration filesystem[0m.
[[0;32m  OK  [0m] Mounted [0;1;39mKernel Debug File System[0m.
[[0;32m  OK  [0m] Mounted [0;1;39mKernel Trace File System[0m.
[[0;32m  OK  [0m] Finished [0;1;39mCreate List of Static Device Nodes[0m.
[[0;32m  OK  [0m] Finished [0;1;39mMonitoring of LVM… dmeventd or progress polling[0m.
[[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Module configfs[0m.
[[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Module drm[0m.
[[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Module efi_pstore[0m.
[[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Module fuse[0m.
         Mounting [0;1;39mFUSE Control File System[0m...
         Mounting [0;1;39mKernel Configuration File System[0m...
[[0;32m  OK  [0m] Mounted [0;1;39mKernel Configuration File System[0m.
[[0;32m  OK  [0m] Mounted [0;1;39mFUSE Control File System[0m.
[[0;32m  OK  [0m] Finished [0;1;39mSet the console keyboard layout[0m.
[    4.546387] EXT4-fs (vda1): re-mounted. Opts: (null). Quota mode: none.
[[0;32m  OK  [0m] Finished [0;1;39mRemount Root and Kernel File Systems[0m.
         Activating swap [0;1;39m/swap.img[0m...
         Starting [0;1;39mDevice-Mapper Multipath Device Controller[0m...
         Starting [0;1;39mFlush Journal to Persistent Storage[0m...
[    4.573612] alua: device handler registered
         Starting [0;1;39mLoad/Save Random Seed[0m...
         Starting [0;1;39mCreate System Users[0m...
[[0;32m  OK  [0m] Finished [0;1;39mLoad Kernel Modules[0m.
         Starting [0;1;39mApply Kernel Variables[0m...
[    4.603498] Adding 4194300k swap on /swap.img.  Priority:-2 extents:9 across:15695872k FS
[    4.607932] emc: device handler registered
[[0;32m  OK  [0m] Activated swap [0;1;39m/swap.img[0m.
[    4.614596] systemd-journald[354]: Received client request to flush runtime journal.
[[0;32m  OK  [0m] Reached target [0;1;39mSwaps[0m.
[    4.619407] rdac: device handler registered
[[0;32m  OK  [0m] Finished [0;1;39mCreate System Users[0m.
         Starting [0;1;39mCreate Static Device Nodes in /dev[0m...
[[0;32m  OK  [0m] Finished [0;1;39mColdplug All udev Devices[0m.
         Starting [0;1;39mHelper to synchronize boot up for ifupdown[0m...
[[0;32m  OK  [0m] Finished [0;1;39mApply Kernel Variables[0m.
[[0;32m  OK  [0m] Finished [0;1;39mCreate Static Device Nodes in /dev[0m.
         Starting [0;1;39mRule-based Manage…for Device Events and Files[0m...
[[0;32m  OK  [0m] Started [0;1;39mDevice-Mapper Multipath Device Controller[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mPreparation for Local File Systems[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mLocal File Systems[0m.
         Starting [0;1;39mLoad AppArmor profiles[0m...
         Starting [0;1;39mSet console font and keymap[0m...
         Starting [0;1;39mCreate final runt…dir for shutdown pivot root[0m...
         Starting [0;1;39mTell Plymouth To Write Out Runtime Data[0m...
         Starting [0;1;39mSet Up Additional Binary Formats[0m...
         Starting [0;1;39mUncomplicated firewall[0m...
[[0;32m  OK  [0m] Finished [0;1;39mLoad/Save Random Seed[0m.
[[0;32m  OK  [0m] Finished [0;1;39mSet console font and keymap[0m.
[[0;32m  OK  [0m] Finished [0;1;39mCreate final runt…e dir for shutdown pivot root[0m.
[[0;32m  OK  [0m] Finished [0;1;39mTell Plymouth To Write Out Runtime Data[0m.
[[0;32m  OK  [0m] Finished [0;1;39mUncomplicated firewall[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mPreparation for Network[0m.
         Mounting [0;1;39mArbitrary Executable File Formats File System[0m...
[[0;32m  OK  [0m] Finished [0;1;39mFlush Journal to Persistent Storage[0m.
         Starting [0;1;39mCreate Volatile Files and Directories[0m...
[    5.712779] audit: type=1400 audit(1734391869.492:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lsb_release" pid=423 comm="apparmor_parser"
[    5.721395] audit: type=1400 audit(1734391869.500:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=424 comm="apparmor_parser"
[    5.724247] audit: type=1400 audit(1734391869.500:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=424 comm="apparmor_parser"
[[0;32m  OK  [0m] Mounted [0;1;39mArbitrary Executable File Formats File System[0m.
[[0;32m  OK  [0m] Finished [0;1;39mSet Up Additional Binary Formats[0m.
[[0;32m  OK  [0m] Finished [0;1;39mCreate Volatile Files and Directories[0m.
[    5.741262] audit: type=1400 audit(1734391869.516:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="ubuntu_pro_apt_news" pid=429 comm="apparmor_parser"
[    5.744148] audit: type=1400 audit(1734391869.520:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=428 comm="apparmor_parser"
[    5.747986] audit: type=1400 audit(1734391869.520:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=428 comm="apparmor_parser"
         Mounting [0;1;39mRPC Pipe File System[0m...
[    5.754143] audit: type=1400 audit(1734391869.520:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=428 comm="apparmor_parser"
[    5.757831] audit: type=1400 audit(1734391869.520:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/{,usr/}sbin/dhclient" pid=428 comm="apparmor_parser"
         Starting [0;1;39mRPC bind portmap service[0m...
         Starting [0;1;39mRecord System Boot/Shutdown in UTMP[0m...
[[0;32m  OK  [0m] Started [0;1;39mRPC bind portmap service[0m.
[[0;32m  OK  [0m] Mounted [0;1;39mRPC Pipe File System[0m.
[    5.776761] audit: type=1400 audit(1734391869.556:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=431 comm="apparmor_parser"
[    5.781109] audit: type=1400 audit(1734391869.556:11): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=431 comm="apparmor_parser"
[[0;32m  OK  [0m] Reached target [0;1;39mrpc_pipefs.target[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mRPC Port Mapper[0m.
         Starting [0;1;39mpNFS block layout mapping daemon[0m...
         Starting [0;1;39mNFSv4 ID-name mapping service[0m...
         Starting [0;1;39mNFSv4 Client Tracking Daemon[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mNFS client services[0m.
[[0;32m  OK  [0m] Finished [0;1;39mRecord System Boot/Shutdown in UTMP[0m.
[[0;32m  OK  [0m] Started [0;1;39mpNFS block layout mapping daemon[0m.
[[0;32m  OK  [0m] Started [0;1;39mNFSv4 ID-name mapping service[0m.
[[0;32m  OK  [0m] Started [0;1;39mNFSv4 Client Tracking Daemon[0m.
[[0;32m  OK  [0m] Started [0;1;39mRule-based Manager for Device Events and Files[0m.
[[0;32m  OK  [0m] Finished [0;1;39mLoad AppArmor profiles[0m.
[[0;32m  OK  [0m] Started [0;1;39mDispatch Password …ts to Console Directory Watch[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mLocal Encrypted Volumes[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mSystem Initialization[0m.
[[0;32m  OK  [0m] Started [0;1;39mDaily Cleanup of Temporary Directories[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mPath Units[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mD-Bus System Message Bus Socket[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mOpen-iSCSI iscsid Socket[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mUUID daemon activation socket[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mSocket Units[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mBasic System[0m.
[[0;32m  OK  [0m] Started [0;1;39mD-Bus System Message Bus[0m.
         Starting [0;1;39mNetwork Manager[0m...
[[0;32m  OK  [0m] Started [0;1;39mSave initial kernel messages after boot[0m.
         Starting [0;1;39mRemove Stale Onli…t4 Metadata Check Snapshots[0m...
         Starting [0;1;39mRecord successful boot for GRUB[0m...
[[0;32m  OK  [0m] Started [0;1;39mirqbalance daemon[0m.
         Starting [0;1;39mKernel crash dump capture service[0m...
         Starting [0;1;39mLSB: Execute the …-e command to reboot system[0m...
         Starting [0;1;39mDispatcher daemon for systemd-networkd[0m...
         Starting [0;1;39mAuthorization Manager[0m...
         Starting [0;1;39mSystem Logging Service[0m...
         Starting [0;1;39mUser Login Management[0m...
         Starting [0;1;39mNetwork Configuration[0m...
         Starting [0;1;39mDisk Manager[0m...
         Starting [0;1;39mWPA supplicant[0m...
[[0;32m  OK  [0m] Finished [0;1;39mRemove Stale Onli…ext4 Metadata Check Snapshots[0m.
[[0;32m  OK  [0m] Started [0;1;39mSystem Logging Service[0m.
[[0;32m  OK  [0m] Started [0;1;39mAuthorization Manager[0m.
[[0;32m  OK  [0m] Started [0;1;39mNetwork Manager[0m.
[[0;32m  OK  [0m] Started [0;1;39mLSB: Execute the k…c -e command to reboot system[0m.
[[0;32m  OK  [0m] Started [0;1;39mUser Login Management[0m.
[[0;32m  OK  [0m] Started [0;1;39mWPA supplicant[0m.
         Starting [0;1;39mModem Manager[0m...
         Starting [0;1;39mNetwork Manager Wait Online[0m...
         Starting [0;1;39mHostname Service[0m...
[[0;32m  OK  [0m] Finished [0;1;39mRecord successful boot for GRUB[0m.
[[0;32m  OK  [0m] Found device [0;1;39m/dev/ttyS0[0m.
[[0;32m  OK  [0m] Started [0;1;39mNetwork Configuration[0m.
         Starting [0;1;39mGRUB failed boot detection[0m...
         Starting [0;1;39mWait for Network to be Configured[0m...
         Starting [0;1;39mNetwork Name Resolution[0m...
[[0;32m  OK  [0m] Started [0;1;39mDispatcher daemon for systemd-networkd[0m.
[    6.444125] kdump-tools[472]: Starting kdump-tools:
[    6.445351] kdump-tools[509]:  * Creating symlink /var/lib/kdump/vmlinuz
[    6.448781] kdump-tools[509]:  * Creating symlink /var/lib/kdump/initrd.img
[[0;32m  OK  [0m] Started [0;1;39mHostname Service[0m.
[[0;32m  OK  [0m] Finished [0;1;39mGRUB failed boot detection[0m.
[[0;32m  OK  [0m] Listening on [0;1;39mLoad/Save RF …itch Status /dev/rfkill Watch[0m.
         Starting [0;1;39mNetwork Manager Script Dispatcher Service[0m...
[[0;32m  OK  [0m] Started [0;1;39mNetwork Manager Script Dispatcher Service[0m.
[[0;32m  OK  [0m] Started [0;1;39mModem Manager[0m.
[[0;32m  OK  [0m] Started [0;1;39mDisk Manager[0m.
[[0;32m  OK  [0m] Started [0;1;39mNetwork Name Resolution[0m.
[[0;32m  OK  [0m] Finished [0;1;39mNetwork Manager Wait Online[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mHost and Network Name Lookups[0m.
[[0;32m  OK  [0m] Found device [0;1;39mVirtio network device[0m.
[[0;32m  OK  [0m] Started [0;1;39mifup for eth0[0m.
[[0;32m  OK  [0m] Finished [0;1;39mHelper to synchronize boot up for ifupdown[0m.
         Starting [0;1;39mRaise network interfaces[0m...
[    7.701780] IPv6: eth0: IPv6 duplicate address fd33:3981:3213:f010:0:5254:ce:6bb5 used by 52:54:00:ba:03:9f detected!
[    7.862786] kdump-tools[509]:  * loaded kdump kernel
[[0;32m  OK  [0m] Finished [0;1;39mKernel crash dump capture service[0m.
[[0m[0;31m*     [0m] (1 of 2) A start job is running for… network interfaces (5s / 5min 2s)
M[K[[0;1;31m*[0m[0;31m*    [0m] (1 of 2) A start job is running for… network interfaces (6s / 5min 2s)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] (1 of 2) A start job is running for… network interfaces (7s / 5min 2s)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] (2 of 2) A start job is running for…k to be Configured (7s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] (2 of 2) A start job is running for…k to be Configured (8s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] (2 of 2) A start job is running for…k to be Configured (8s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] (1 of 2) A start job is running for… network interfaces (9s / 5min 2s)
M[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mRaise network interfaces[0m.
[KSee 'systemctl status networking.service' for details.
[[0;32m  OK  [0m] Reached target [0;1;39mNetwork[0m.
         Starting [0;1;39mchrony, an NTP client/server[0m...
[[0;32m  OK  [0m] Started [0;1;39mUnattended Upgrades Shutdown[0m.
[[0;32m  OK  [0m] Started [0;1;39mchrony, an NTP client/server[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mSystem Time Synchronized[0m.
[[0;32m  OK  [0m] Started [0;1;39mDaily apt download activities[0m.
[[0;32m  OK  [0m] Started [0;1;39mDaily apt upgrade and clean activities[0m.
[[0;32m  OK  [0m] Started [0;1;39mDaily dpkg database backup timer[0m.
[[0;32m  OK  [0m] Started [0;1;39mPeriodic ext4 Onli…ata Check for All Filesystems[0m.
[[0;32m  OK  [0m] Started [0;1;39mDiscard unused blocks once a week[0m.
[[0;32m  OK  [0m] Started [0;1;39mRefresh fwupd metadata regularly[0m.
[[0;32m  OK  [0m] Started [0;1;39mDaily rotation of log files[0m.
[[0;32m  OK  [0m] Started [0;1;39mDaily man-db regeneration[0m.
[[0;32m  OK  [0m] Started [0;1;39mMessage of the Day[0m.
         Starting [0;1;39mMUNGE authentication service[0m...
[[0;32m  OK  [0m] Started [0;1;39mMUNGE authentication service[0m.
[     [0;31m*[0m] A start job is running for Wait for… to be Configured (12s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (13s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (13s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (14s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (14s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (15s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (16s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for… to be Configured (16s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (17s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (17s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (18s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (18s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (19s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (20s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for… to be Configured (20s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (21s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (21s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (22s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (23s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (24s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for… to be Configured (24s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (25s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (25s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (26s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (27s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (27s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (28s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for… to be Configured (28s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (29s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (29s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (30s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (30s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (31s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (31s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (32s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (33s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (33s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (34s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (34s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (35s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for… to be Configured (35s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (36s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (36s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (37s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (37s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (38s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (39s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (40s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (40s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (41s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (41s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (42s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for… to be Configured (42s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (43s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (43s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (44s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (44s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (45s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (46s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (47s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (47s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (48s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (48s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (49s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for… to be Configured (49s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (50s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (50s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (51s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (51s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (52s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (53s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (54s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (54s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (55s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (55s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (56s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for… to be Configured (56s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for… to be Configured (57s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for… to be Configured (57s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for… to be Configured (58s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for… to be Configured (58s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for… to be Configured (59s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…to be Configured (1min / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…be Configured (1min 1s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 2s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…be Configured (1min 3s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 4s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…be Configured (1min 5s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…be Configured (1min 6s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…be Configured (1min 7s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…be Configured (1min 8s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…be Configured (1min 9s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 10s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 11s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 12s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 13s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 14s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 15s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 16s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 17s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 18s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 19s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 20s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 21s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 22s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 23s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 24s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 25s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 26s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 27s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 28s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 29s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 30s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 31s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 32s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 33s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 34s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 35s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 36s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 37s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 38s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 39s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 40s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 41s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 42s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 43s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 44s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 45s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 46s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 47s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 48s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 49s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 50s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 51s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 52s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 53s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 54s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 55s / no limit)
M[K[[0m[0;31m*     [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M[K[[0;1;31m*[0m[0;31m*    [0m] A start job is running for Wait for…e Configured (1min 56s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…e Configured (1min 57s / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…e Configured (1min 58s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M[K[     [0;31m*[0m] A start job is running for Wait for…e Configured (1min 59s / no limit)
M[K[    [0;31m*[0;1;31m*[0m] A start job is running for Wait for…to be Configured (2min / no limit)
M[K[   [0;31m*[0;1;31m*[0m[0;31m*[0m] A start job is running for Wait for…to be Configured (2min / no limit)
M[K[  [0;31m*[0;1;31m*[0m[0;31m* [0m] A start job is running for Wait for…be Configured (2min 1s / no limit)
M[K[ [0;31m*[0;1;31m*[0m[0;31m*  [0m] A start job is running for Wait for…be Configured (2min 1s / no limit)
M[K[[0;31m*[0;1;31m*[0m[0;31m*   [0m] A start job is running for Wait for…be Configured (2min 2s / no limit)
M[K[[0;1;31mFAILED[0m] Failed to start [0;1;39mWait for Network to be Configured[0m.
[KSee 'systemctl status systemd-networkd-wait-online.service' for details.
[[0;32m  OK  [0m] Reached target [0;1;39mNetwork is Online[0m.
[[0;32m  OK  [0m] Started [0;1;39mDownload data for …ailed at package install time[0m.
[[0;32m  OK  [0m] Started [0;1;39mCheck to see wheth…w version of Ubuntu available[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mTimer Units[0m.
         Starting [0;1;39mNFS Mount Daemon[0m...
         Starting [0;1;39mSamba NMB Daemon[0m...
[[0;32m  OK  [0m] Reached target [0;1;39mPreparation for Remote File Systems[0m.
         Mounting [0;1;39m/autotest[0m...
         Mounting [0;1;39m/scratch[0m...
[[0;32m  OK  [0m] Finished [0;1;39mAvailability of block devices[0m.
[  126.680311] FS-Cache: Loaded
         Starting [0;1;39mPostfix Mail Transport Agent (instance -)[0m...[  126.701979] FS-Cache: Netfs 'nfs' registered for caching

         Starting [0;1;39mNFS status monitor for NFSv2/3 locking.[0m...
         Starting [0;1;39mOpenBSD Secure Shell server[0m...
[[0;32m  OK  [0m] Started [0;1;39mNFS Mount Daemon[0m.
[[0;32m  OK  [0m] Started [0;1;39mNFS status monitor for NFSv2/3 locking.[0m.
[[0;32m  OK  [0m] Started [0;1;39mSamba NMB Daemon[0m.
         Starting [0;1;39mNFS server and services[0m...
         Starting [0;1;39mSamba SMB Daemon[0m...
[  126.906849] NFS: Registering the id_resolver key type
[  126.908289] Key type id_resolver registered
[  126.909931] Key type id_legacy registered
[[0;32m  OK  [0m] Started [0;1;39mOpenBSD Secure Shell server[0m.
[  126.919545] FS-Cache: Duplicate cookie detected
[  126.920491] FS-Cache: O-cookie c=00000003 [p=00000002 fl=222 nc=0 na=1]
[  126.922065] FS-Cache: O-cookie d=000000001de76d3d{NFS.server} n=000000003dfdc495
[  126.923462] FS-Cache: O-key=[16] '0400000002000000020008010af010cc'
[  126.924921] FS-Cache: N-cookie c=00000004 [p=00000002 fl=2 nc=0 na=1]
[  126.924923] FS-Cache: N-cookie d=000000001de76d3d{NFS.server} n=00000000ea719c55
[  126.924926] FS-Cache: N-key=[16] '0400000002000000020008010af010cc'
[[0;32m  OK  [0m] Started [0;1;39mSamba SMB Daemon[0m.
[[0;32m  OK  [0m] Mounted [0;1;39m/autotest[0m.
[[0;32m  OK  [0m] Mounted [0;1;39m/scratch[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mRemote File Systems[0m.
         Starting [0;1;39mLSB: automatic crash report generation[0m...
         Starting [0;1;39mAutomounts filesystems on demand[0m...
[[0;32m  OK  [0m] Started [0;1;39mRegular background program processing daemon[0m.
         Starting [0;1;39mLSB: Load kernel image with kexec[0m...
         Starting [0;1;39mLSB: Start/Stop t…mote power manager) daemon.[0m...
         Starting [0;1;39mPermit User Sessions[0m...
         Starting [0;1;39mLSB: Start vmtouch daemon[0m...
         Starting [0;1;39mLSB: Starts or stops the xinetd daemon.[0m...
[[0;32m  OK  [0m] Started [0;1;39mLSB: Load kernel image with kexec[0m.
[  130.502403] NFSD: Using nfsdcld client tracking operations.
[  130.503591] NFSD: no clients to reclaim, skipping NFSv4 grace period (net f0000000)
[[0;32m  OK  [0m] Finished [0;1;39mPermit User Sessions[0m.
[[0;32m  OK  [0m] Started [0;1;39mLSB: automatic crash report generation[0m.
         Starting [0;1;39mHold until boot process finishes up[0m...
         Starting [0;1;39mTerminate Plymouth Boot Screen[0m...
[[0;32m  OK  [0m] Finished [0;1;39mHold until boot process finishes up[0m.
[[0;32m  OK  [0m] Started [0;1;39mSerial Getty on ttyS0[0m.
         Starting [0;1;39mSet console scheme[0m...
[[0;32m  OK  [0m] Started [0;1;39mLSB: Start vmtouch daemon[0m.
[[0;32m  OK  [0m] Finished [0;1;39mTerminate Plymouth Boot Screen[0m.
[[0;32m  OK  [0m] Finished [0;1;39mSet console scheme[0m.
[[0;32m  OK  [0m] Created slice [0;1;39mSlice /system/getty[0m.
[[0;32m  OK  [0m] Started [0;1;39mGetty on tty1[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mLogin Prompts[0m.
[[0;32m  OK  [0m] Started [0;1;39mLSB: Start/Stop th…remote power manager) daemon.[0m.
[[0;32m  OK  [0m] Finished [0;1;39mNFS server and services[0m.
         Starting [0;1;39mNotify NFS peers of a restart[0m...
[[0;32m  OK  [0m] Started [0;1;39mNotify NFS peers of a restart[0m.
[[0;32m  OK  [0m] Started [0;1;39mAutomounts filesystems on demand[0m.
[[0;32m  OK  [0m] Created slice [0;1;39mUser Slice of UID 0[0m.
         Starting [0;1;39mUser Runtime Directory /run/user/0[0m...
[[0;32m  OK  [0m] Started [0;1;39mLSB: Starts or stops the xinetd daemon.[0m.
[[0;32m  OK  [0m] Finished [0;1;39mUser Runtime Directory /run/user/0[0m.
         Starting [0;1;39mUser Manager for UID 0[0m...
         Starting [0;1;39mUpdate APT News[0m...
         Starting [0;1;39mUpdate the local ESM caches[0m...
[[0;32m  OK  [0m] Started [0;1;39mUser Manager for UID 0[0m.
[[0;32m  OK  [0m] Started [0;1;39mSession 3 of User root[0m.
[[0;32m  OK  [0m] Finished [0;1;39mUpdate APT News[0m.
[[0;32m  OK  [0m] Started [0;1;39mPostfix Mail Transport Agent (instance -)[0m.
         Starting [0;1;39mPostfix Mail Transport Agent[0m...
[[0;32m  OK  [0m] Finished [0;1;39mPostfix Mail Transport Agent[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mMulti-User System[0m.
[[0;32m  OK  [0m] Reached target [0;1;39mGraphical Interface[0m.
         Starting [0;1;39mRecord Runlevel Change in UTMP[0m...
[[0;32m  OK  [0m] Finished [0;1;39mRecord Runlevel Change in UTMP[0m.
[  136.123004] kauditd_printk_skb: 12 callbacks suppressed
[  136.123008] audit: type=1400 audit(1734391999.907:24): apparmor="DENIED" operation="open" profile="ubuntu_pro_esm_cache//apt_methods_gpgv" name="/dev/tty" pid=1905 comm="apt-key" requested_mask="wr" denied_mask="wr" fsuid=100 ouid=0
[  136.183244] audit: type=1400 audit(1734391999.967:25): apparmor="DENIED" operation="open" profile="ubuntu_pro_esm_cache//apt_methods_gpgv" name="/dev/tty" pid=1955 comm="apt-key" requested_mask="wr" denied_mask="wr" fsuid=100 ouid=0

Ubuntu 22.04.5 LTS onyx-138vm1.onyx.whamcloud.com ttyS0

onyx-138vm1 login: [  165.583843] Autotest: Nodes configured in 3 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  167.145078] Autotest: Node provisioning/configuration took 29 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  168.560367] Autotest: Starting test framework on controller onyx-138vm1 (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  174.972501] Autotest: Test framework started (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  175.238186] Autotest: Test framework pid=2741 (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  175.563848] Autotest: Test running for 0 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  175.834265] libcfs: loading out-of-tree module taints kernel.
[  175.835809] libcfs: module verification failed: signature and/or required key missing - tainting kernel
[  175.838979] Key type ._llcrypt registered
[  175.839901] Key type .llcrypt registered
[  175.862493] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[  179.083726] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[  179.098603] Lustre: DEBUG MARKER: /usr/sbin/lctl lustre_build_version 2>/dev/null
[  182.007133] Lustre: DEBUG MARKER: cat /etc/system-release
[  182.018718] Lustre: DEBUG MARKER: test -r /etc/os-release
[  182.029572] Lustre: DEBUG MARKER: cat /etc/os-release
[  182.416727] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[  182.416727] if [ $running -ne 0 ] ; then
[  182.416727] echo Stopping client $(hostname) /mnt/lustre opts:-f;
[  182.416727] lsof /mnt/lustre || need_kill=no;
[  182.416727] if [ x-f != x -a x$need_kill != xno ]; then
[  182.416727]     pids=$(lsof -t /mnt/lustre | sort -u);
[  182.416727]    
[  183.098205] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[  183.098205] if [ $running -ne 0 ] ; then
[  183.098205] echo Stopping client $(hostname) /mnt/lustre2 opts:-f;
[  183.098205] lsof /mnt/lustre2 || need_kill=no;
[  183.098205] if [ x-f != x -a x$need_kill != xno ]; then
[  183.098205]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[  196.657559] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[  196.759871] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[  196.761116] LNet: Accept all, port 7988
[  199.235227] Key type lgssc registered
[  199.383554] Lustre: Echo OBD driver; http://www.lustre.org/
[  267.214273] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[  267.214273] mpts=$(mount | grep -c /mnt/lustre' ');
[  267.214273] if [ $running -ne $mpts ]; then
[  267.214273]     echo $(hostname) env are INSANE!;
[  267.214273]     exit 1;
[  267.214273] fi
[  267.900019] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[  267.900019] mpts=$(mount | grep -c /mnt/lustre2' ');
[  267.900019] if [ $running -ne $mpts ]; then
[  267.900019]     echo $(hostname) env are INSANE!;
[  267.900019]     exit 1;
[  267.900019] fi
[  357.082839] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[  357.092643] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[  358.127789] Lustre: Mounted lustre-client
[  358.476902] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[  359.161048] Lustre: DEBUG MARKER: 
[  359.161048] running=$(mount | grep -c /mnt/lustre' ');
[  359.161048] rc=0;
[  359.161048] if [ $running -eq 0 ] ; then
[  359.161048] 	mkdir -p /mnt/lustre;
[  359.161048] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[  359.161048] 	rc=$?;
[  359.161048] fi;
[  359.161048] exit $rc
[  366.093277] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[  375.893853] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[  376.251122] Lustre: DEBUG MARKER: Using TIMEOUT=20
[  381.650347] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[  382.329393] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[  382.355516] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[  382.369186] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[  382.693222] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[  383.707005] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[  384.720271] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[  385.062812] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[  385.072807] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[  393.009349] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/li
[  394.588508] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing check_logdir \/autotest\/autotest-2\/2024-12-16\/lustre-reviews_review-ldiskfs-ubuntu_109661_43_0db25c81-f1b5-4d06-b429-66da95d8edf2\/
[  395.177294] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing check_logdir /autotest/autotest-2/2024-12-16/lustre-reviews_review-ldiskfs-ubuntu_109661_43_0db25c81-f1b5-4d06-b429-66da95d8edf2/
[  395.927840] Lustre: DEBUG MARKER: uname -n
[  397.561357] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/li
[  399.014136] Lustre: lustre-OST0000-osc-ffff988d0505d000: disconnect after 24s idle
[  399.149920] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing yml_node
[  399.855456] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing yml_node
[  400.325229] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[  400.895082] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[  401.311890] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Client: 2.16.50.134
[  401.673312] Lustre: DEBUG MARKER: Client: 2.16.50.134
[  402.396605] Lustre: DEBUG MARKER: /usr/sbin/lctl mark MDS: 2.16.50.134
[  402.758361] Lustre: DEBUG MARKER: MDS: 2.16.50.134
[  403.514400] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OSS: 2.16.50.134
[  403.881375] Lustre: DEBUG MARKER: OSS: 2.16.50.134
[  404.301226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: lnet-selftest ============----- Mon Dec 16 23:37:47 UTC 2024
[  404.658349] Lustre: DEBUG MARKER: -----============= acceptance-small: lnet-selftest ============----- Mon Dec 16 23:37:47 UTC 2024
[  406.052679] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[  408.872001] Lustre: DEBUG MARKER: cat /etc/system-release
[  408.883715] Lustre: DEBUG MARKER: test -r /etc/os-release
[  408.894650] Lustre: DEBUG MARKER: cat /etc/os-release
[  409.993387] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests: 
[  410.350797] Lustre: DEBUG MARKER: excepting tests:
[  420.872409] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[  421.733531] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[  421.733531] if [ $running -ne 0 ] ; then
[  421.733531] echo Stopping client $(hostname) /mnt/lustre opts:;
[  421.733531] lsof /mnt/lustre || need_kill=no;
[  421.733531] if [ x != x -a x$need_kill != xno ]; then
[  421.733531]     pids=$(lsof -t /mnt/lustre | sort -u);
[  421.733531]     if 
[  422.878371] Lustre: Unmounted lustre-client
[  423.322786] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == lnet-selftest test smoke: lst regression test ========= 23:38:06 \(1734392286\)
[  423.682657] Lustre: DEBUG MARKER: == lnet-selftest test smoke: lst regression test ========= 23:38:06 (1734392286)
[  424.175216] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[  425.572322] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing lst_cleanup
[  426.059785] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing lst_cleanup
[  427.615403] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[  429.201945] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing lst_setup
[  429.852977] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing lst_setup
[  431.481334] LNet: 14033:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  431.586034] LNet: 14033:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  431.690023] LNet: 14033:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  431.794026] LNet: 14033:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  431.898036] LNet: 14033:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  432.002041] LNet: 14033:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  432.106028] LNet: 14033:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  432.518862] LNet: 9:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  432.735098] LNet: 9:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  432.953143] LNet: 102:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  433.182789] LNet: 102:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  433.398880] LNet: 106:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  433.615180] LNet: 106:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  433.834631] LNet: 9:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  434.072606] LNet: 9:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  434.294103] LNet: 106:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  434.517678] LNet: 106:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  434.735184] LNet: 106:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  434.951361] LNet: 106:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[  476.062462] Autotest: Test running for 5 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  736.476538] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[  738.037874] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing lst_cleanup
[  738.601243] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing lst_cleanup
[  739.184489] LNet: 14507:0:(timer.c:220:stt_shutdown()) waiting for 1 threads to terminate
[  741.262319] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == lnet-selftest test complete, duration 336 sec ========= 23:43:24 \(1734392604\)
[  741.620075] Lustre: DEBUG MARKER: == lnet-selftest test complete, duration 336 sec ========= 23:43:24 (1734392604)
[  741.961587] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[  742.645696] Lustre: DEBUG MARKER: 
[  742.645696] running=$(mount | grep -c /mnt/lustre' ');
[  742.645696] rc=0;
[  742.645696] if [ $running -eq 0 ] ; then
[  742.645696] 	mkdir -p /mnt/lustre;
[  742.645696] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[  742.645696] 	rc=$?;
[  742.645696] fi;
[  742.645696] exit $rc
[  743.041639] Lustre: Mounted lustre-client
[  743.384476] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[  746.111686] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === lnet-selftest: start cleanup 23:43:29 \(1734392609\) ===
[  746.468256] Lustre: DEBUG MARKER: === lnet-selftest: start cleanup 23:43:29 (1734392609) ===
[  747.323950] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === lnet-selftest: finish cleanup 23:43:30 \(1734392610\) ===
[  747.680154] Lustre: DEBUG MARKER: === lnet-selftest: finish cleanup 23:43:30 (1734392610) ===
[  748.111174] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: sanity ============----- Mon Dec 16 23:43:31 UTC 2024
[  748.470029] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity ============----- Mon Dec 16 23:43:31 UTC 2024
[  749.854774] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[  752.631879] Lustre: DEBUG MARKER: cat /etc/system-release
[  752.643617] Lustre: DEBUG MARKER: test -r /etc/os-release
[  752.654821] Lustre: DEBUG MARKER: cat /etc/os-release
[  753.787633] Lustre: DEBUG MARKER: /usr/sbin/lctl mark - need mds1 \< v2_14_55-100-g8a84c7f9c7 for LU-14927, skip 0f
[  754.145882] Lustre: DEBUG MARKER: - need mds1 < v2_14_55-100-g8a84c7f9c7 for LU-14927, skip 0f
[  754.547374] Lustre: DEBUG MARKER: /usr/sbin/lctl mark - need mds1 \<= 2.14.55-100-g8a84c7f9c7 for LU-14927, skip 0f
[  754.904599] Lustre: DEBUG MARKER: - need mds1 <= 2.14.55-100-g8a84c7f9c7 for LU-14927, skip 0f
[  755.307224] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests: 0f 0f 0f 42a 42c 42b 118c 118d 407 119i 817
[  755.664033] Lustre: DEBUG MARKER: excepting tests: 0f 0f 0f 42a 42c 42b 118c 118d 407 119i 817
[  756.069076] Lustre: DEBUG MARKER: /usr/sbin/lctl mark skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842
[  756.422780] Lustre: DEBUG MARKER: skipping tests SLOW=no: 27m 60i 64b 68 71 135 136 230d 300o 842
[  756.830722] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: start setup 23:43:40 \(1734392620\) ===
[  757.187158] Lustre: DEBUG MARKER: === sanity: start setup 23:43:40 (1734392620) ===
[  757.548369] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[  758.792508] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing check_config_client \/mnt\/lustre
[  759.276117] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing check_config_client /mnt/lustre
[  762.979822] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[  762.979822] mpts=$(mount | grep -c /mnt/lustre' ');
[  762.979822] if [ $running -ne $mpts ]; then
[  762.979822]     echo $(hostname) env are INSANE!;
[  762.979822]     exit 1;
[  762.979822] fi
[  763.670505] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[  763.670505] mpts=$(mount | grep -c /mnt/lustre2' ');
[  763.670505] if [ $running -ne $mpts ]; then
[  763.670505]     echo $(hostname) env are INSANE!;
[  763.670505]     exit 1;
[  763.670505] fi
[  770.806550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[  771.166086] Lustre: DEBUG MARKER: Using TIMEOUT=20
[  771.561552] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[  772.264147] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[  776.908690] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: finish setup 23:44:00 \(1734392640\) ===
[  777.086409] Autotest: Test running for 10 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[  777.305128] Lustre: DEBUG MARKER: === sanity: finish setup 23:44:00 (1734392640) ===
[  778.186191] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0a: touch\; rm ============================= 23:44:01 \(1734392641\)
[  778.545471] Lustre: DEBUG MARKER: == sanity test 0a: touch; rm ============================= 23:44:01 (1734392641)
[  780.390766] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0b: chmod 0755 \/mnt\/lustre ======================================================================================= 23:44:03 \(1734392643\)
[  780.749799] Lustre: DEBUG MARKER: == sanity test 0b: chmod 0755 /mnt/lustre ======================================================================================= 23:44:03 (1734392643)
[  782.483001] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0c: check import proc ===================== 23:44:05 \(1734392645\)
[  782.844791] Lustre: DEBUG MARKER: == sanity test 0c: check import proc ===================== 23:44:05 (1734392645)
[  784.581967] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0d: check export proc ======================================================================================= 23:44:08 \(1734392648\)
[  784.940605] Lustre: DEBUG MARKER: == sanity test 0d: check export proc ======================================================================================= 23:44:08 (1734392648)
[  786.196323] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[  787.925016] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 0e: Enable DNE MDT balancing for mkdir in the ROOT ========================================================== 23:44:11 \(1734392651\)
[  788.285805] Lustre: DEBUG MARKER: == sanity test 0e: Enable DNE MDT balancing for mkdir in the ROOT ========================================================== 23:44:11 (1734392651)
[  788.693644] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_0e We need at least 2 MDTs for this test
[  789.048813] Lustre: DEBUG MARKER: SKIP: sanity test_0e We need at least 2 MDTs for this test
[  789.473095] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 1: mkdir\; remkdir\; rmdir ================== 23:44:12 \(1734392652\)
[  789.837195] Lustre: DEBUG MARKER: == sanity test 1: mkdir; remkdir; rmdir ================== 23:44:12 (1734392652)
[  791.591891] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 2: mkdir\; touch\; rmdir\; check file ======== 23:44:15 \(1734392655\)
[  791.953700] Lustre: DEBUG MARKER: == sanity test 2: mkdir; touch; rmdir; check file ======== 23:44:15 (1734392655)
[  793.795972] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 3: mkdir\; touch\; rmdir\; check dir ========= 23:44:17 \(1734392657\)
[  794.157638] Lustre: DEBUG MARKER: == sanity test 3: mkdir; touch; rmdir; check dir ========= 23:44:17 (1734392657)
[  796.021539] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 4: mkdir\; touch dir\/file\; rmdir\; checkdir \(expect error\) ========================================================== 23:44:19 \(1734392659\)
[  796.388043] Lustre: DEBUG MARKER: == sanity test 4: mkdir; touch dir/file; rmdir; checkdir (expect error) ========================================================== 23:44:19 (1734392659)
[  798.262022] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 5: mkdir ...\/d5 ...\/d5\/d2\; chmod ...\/d5\/d2 ========================================================== 23:44:21 \(1734392661\)
[  798.623789] Lustre: DEBUG MARKER: == sanity test 5: mkdir .../d5 .../d5/d2; chmod .../d5/d2 ========================================================== 23:44:21 (1734392661)
[  799.395643] Lustre: lustre-OST0001-osc-ffff988d17847800: disconnect after 22s idle
[  799.397240] Lustre: Skipped 6 previous similar messages
[  800.394646] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6a: touch f6a\; chmod f6a\; runas -u 500 -g 500 chmod f6a \(should return error\) ============================================================ 23:44:23 \(1734392663\)
[  800.758586] Lustre: DEBUG MARKER: == sanity test 6a: touch f6a; chmod f6a; runas -u 500 -g 500 chmod f6a (should return error) ============================================================ 23:44:23 (1734392663)
[  802.530978] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6c: touch f6c\; chown f6c\; runas -u 500 -g 500 chown f6c \(should return error\) ============================================================ 23:44:25 \(1734392665\)
[  802.896271] Lustre: DEBUG MARKER: == sanity test 6c: touch f6c; chown f6c; runas -u 500 -g 500 chown f6c (should return error) ============================================================ 23:44:25 (1734392665)
[  804.647148] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6e: touch+chgrp \; runas -u 500 -g 500 chgrp  \(should return error\) ========================================================== 23:44:28 \(1734392668\)
[  805.006318] Lustre: DEBUG MARKER: == sanity test 6e: touch+chgrp ; runas -u 500 -g 500 chgrp (should return error) ========================================================== 23:44:28 (1734392668)
[  806.759084] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6g: verify new dir in sgid dir inherits group ========================================================== 23:44:30 \(1734392670\)
[  807.118959] Lustre: DEBUG MARKER: == sanity test 6g: verify new dir in sgid dir inherits group ========================================================== 23:44:30 (1734392670)
[  808.875039] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 6h: runas -u 500 -g 500 chown RUNAS_ID.0 ...\/ \(should return error\) ========================================================== 23:44:32 \(1734392672\)
[  809.236334] Lustre: DEBUG MARKER: == sanity test 6h: runas -u 500 -g 500 chown RUNAS_ID.0 .../ (should return error) ========================================================== 23:44:32 (1734392672)
[  810.979209] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 7a: mkdir ...\/d7\; mcreate ...\/d7\/f\; chmod ...\/d7\/f ============================================================== 23:44:34 \(1734392674\)
[  811.341850] Lustre: DEBUG MARKER: == sanity test 7a: mkdir .../d7; mcreate .../d7/f; chmod .../d7/f ============================================================== 23:44:34 (1734392674)
[  813.081238] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 7b: mkdir ...\/d7\; mcreate d7\/f2\; echo foo \> d7\/f2 =============================================================== 23:44:36 \(1734392676\)
[  813.437713] Lustre: DEBUG MARKER: == sanity test 7b: mkdir .../d7; mcreate d7/f2; echo foo > d7/f2 =============================================================== 23:44:36 (1734392676)
[  815.186411] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 8: mkdir ...\/d8\; touch ...\/d8\/f\; chmod ...\/d8\/f ================================================================= 23:44:38 \(1734392678\)
[  815.548822] Lustre: DEBUG MARKER: == sanity test 8: mkdir .../d8; touch .../d8/f; chmod .../d8/f ================================================================= 23:44:38 (1734392678)
[  817.284738] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 9: mkdir ...\/d9 ...\/d9\/d2 ...\/d9\/d2\/d3 ========================================================================== 23:44:40 \(1734392680\)
[  817.646757] Lustre: DEBUG MARKER: == sanity test 9: mkdir .../d9 .../d9/d2 .../d9/d2/d3 ========================================================================== 23:44:40 (1734392680)
[  819.406056] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 10: mkdir ...\/d10 ...\/d10\/d2\; touch ...\/d10\/d2\/f ================================================================ 23:44:42 \(1734392682\)
[  819.768414] Lustre: DEBUG MARKER: == sanity test 10: mkdir .../d10 .../d10/d2; touch .../d10/d2/f ================================================================ 23:44:42 (1734392682)
[  821.522714] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 11: mkdir ...\/d11 d11\/d2\; chmod ...\/d11\/d2 ====================================================================== 23:44:44 \(1734392684\)
[  821.885469] Lustre: DEBUG MARKER: == sanity test 11: mkdir .../d11 d11/d2; chmod .../d11/d2 ====================================================================== 23:44:44 (1734392684)
[  823.649479] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 12: touch ...\/d12\/f\; chmod ...\/d12\/f ...\/d12\/f ================================================================== 23:44:47 \(1734392687\)
[  824.010198] Lustre: DEBUG MARKER: == sanity test 12: touch .../d12/f; chmod .../d12/f .../d12/f ================================================================== 23:44:47 (1734392687)
[  825.781612] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 13: creat ...\/d13\/f\; dd ...\/d13\/f\; \> ...\/d13\/f ================================================================== 23:44:49 \(1734392689\)
[  826.146175] Lustre: DEBUG MARKER: == sanity test 13: creat .../d13/f; dd .../d13/f; > .../d13/f ================================================================== 23:44:49 (1734392689)
[  827.907167] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 14: touch ...\/d14\/f\; rm ...\/d14\/f\; rm ...\/d14\/f ================================================================= 23:44:51 \(1734392691\)
[  828.267716] Lustre: DEBUG MARKER: == sanity test 14: touch .../d14/f; rm .../d14/f; rm .../d14/f ================================================================= 23:44:51 (1734392691)
[  830.050110] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 15: touch ...\/d15\/f\; mv ...\/d15\/f ...\/d15\/f2 ==================================================================== 23:44:53 \(1734392693\)
[  830.412328] Lustre: DEBUG MARKER: == sanity test 15: touch .../d15/f; mv .../d15/f .../d15/f2 ==================================================================== 23:44:53 (1734392693)
[  832.171583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 16: touch ...\/d16\/f\; rm -rf ...\/d16\/f ===== 23:44:55 \(1734392695\)
[  832.531975] Lustre: DEBUG MARKER: == sanity test 16: touch .../d16/f; rm -rf .../d16/f ===== 23:44:55 (1734392695)
[  834.282707] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17a: symlinks: create, remove \(real\) ====== 23:44:57 \(1734392697\)
[  834.640009] Lustre: DEBUG MARKER: == sanity test 17a: symlinks: create, remove (real) ====== 23:44:57 (1734392697)
[  836.404226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17b: symlinks: create, remove \(dangling\) == 23:44:59 \(1734392699\)
[  836.764474] Lustre: DEBUG MARKER: == sanity test 17b: symlinks: create, remove (dangling) == 23:44:59 (1734392699)
[  838.520034] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17c: symlinks: open dangling \(should return error\) ========================================================== 23:45:01 \(1734392701\)
[  838.880427] Lustre: DEBUG MARKER: == sanity test 17c: symlinks: open dangling (should return error) ========================================================== 23:45:01 (1734392701)
[  840.634076] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17d: symlinks: create dangling ============ 23:45:04 \(1734392704\)
[  840.994733] Lustre: DEBUG MARKER: == sanity test 17d: symlinks: create dangling ============ 23:45:04 (1734392704)
[  842.734489] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17e: symlinks: create recursive symlink \(should return error\) ========================================================== 23:45:06 \(1734392706\)
[  843.094089] Lustre: DEBUG MARKER: == sanity test 17e: symlinks: create recursive symlink (should return error) ========================================================== 23:45:06 (1734392706)
[  844.835790] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17f: symlinks: long and very long symlink name ========================================================== 23:45:08 \(1734392708\)
[  845.199197] Lustre: DEBUG MARKER: == sanity test 17f: symlinks: long and very long symlink name ========================================================== 23:45:08 (1734392708)
[  847.029824] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17g: symlinks: really long symlink name and inode boundaries ========================================================== 23:45:10 \(1734392710\)
[  847.394437] Lustre: DEBUG MARKER: == sanity test 17g: symlinks: really long symlink name and inode boundaries ========================================================== 23:45:10 (1734392710)
[  849.320745] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17h: create objects: lov_free_memmd\(\) doesn\'t lbug ========================================================== 23:45:12 \(1734392712\)
[  849.727758] Lustre: DEBUG MARKER: == sanity test 17h: create objects: lov_free_memmd() doesn't lbug ========================================================== 23:45:12 (1734392712)
[  851.853971] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17i: don\'t panic on short symlink \(should return error\) ========================================================== 23:45:15 \(1734392715\)
[  852.216179] Lustre: DEBUG MARKER: == sanity test 17i: don't panic on short symlink (should return error) ========================================================== 23:45:15 (1734392715)
[  852.590476] LustreError: 34354:0:(symlink.c:104:ll_readlink_internal()) lustre: inode [0x200000401:0x4f:0x0]: symlink length 33 not expected 35
[  854.338438] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17k: symlinks: rsync with xattrs enabled == 23:45:17 \(1734392717\)
[  854.698738] Lustre: DEBUG MARKER: == sanity test 17k: symlinks: rsync with xattrs enabled == 23:45:17 (1734392717)
[  856.609198] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17l: Ensure lgetxattr\'s returned xattr size is consistent ========================================================== 23:45:20 \(1734392720\)
[  856.969128] Lustre: DEBUG MARKER: == sanity test 17l: Ensure lgetxattr's returned xattr size is consistent ========================================================== 23:45:20 (1734392720)
[  858.770397] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17m: run e2fsck against MDT which contains short\/long symlink ========================================================== 23:45:22 \(1734392722\)
[  859.129792] Lustre: DEBUG MARKER: == sanity test 17m: run e2fsck against MDT which contains short/long symlink ========================================================== 23:45:22 (1734392722)
[  871.082428] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[  871.082435] Lustre: lustre-MDT0000-mdc-ffff988d17847800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[  871.090401] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28ed83e6d1 to 0xdf05ec28ed84bdcb
[  871.092922] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[  871.111614] LustreError: 4379:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988d03a4d6c0 x1818641888745856/t4294967320(4294967320) o101->lustre-MDT0000-mdc-ffff988d17847800@10.240.22.189@tcp:12/10 lens 576/608 e 0 to 0 dl 1734392750 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'grep.0' uid:0 gid:0
[  876.774825] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17n: run e2fsck against master\/slave MDT which contains remote dir ========================================================== 23:45:40 \(1734392740\)
[  877.131015] Lustre: DEBUG MARKER: == sanity test 17n: run e2fsck against master/slave MDT which contains remote dir ========================================================== 23:45:40 (1734392740)
[  877.527802] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_17n needs \>= 2 MDTs
[  877.884114] Lustre: DEBUG MARKER: SKIP: sanity test_17n needs >= 2 MDTs
[  878.310876] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17o: stat file with incompat LMA feature == 23:45:41 \(1734392741\)
[  878.666959] Lustre: DEBUG MARKER: == sanity test 17o: stat file with incompat LMA feature == 23:45:41 (1734392741)
[  881.315868] Lustre: lustre-MDT0000-mdc-ffff988d17847800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[  896.677715] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[  896.681443] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28ed84bdcb to 0xdf05ec28ed85957b
[  896.684076] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[  896.685845] Lustre: Skipped 1 previous similar message
[  901.832948] Lustre: lustre-MDT0000-mdc-ffff988d17847800: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[  902.205736] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[  903.559049] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[  903.998159] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[  904.177121] Lustre: DEBUG MARKER: lctl get_param -n at_max
[  904.576946] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[  905.046021] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[  909.161503] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 17p: symlink overwrite directory error message ========================================================== 23:46:12 \(1734392772\)
[  909.520778] Lustre: DEBUG MARKER: == sanity test 17p: symlink overwrite directory error message ========================================================== 23:46:12 (1734392772)
[  911.336145] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 18: touch ...\/f \; ls ... ======================================================================================== 23:46:14 \(1734392774\)
[  911.698882] Lustre: DEBUG MARKER: == sanity test 18: touch .../f ; ls ... ======================================================================================== 23:46:14 (1734392774)
[  913.439375] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19a: touch ...\/f19 \; ls -l ... \; rm ...\/f19 ===================================================================== 23:46:16 \(1734392776\)
[  913.797765] Lustre: DEBUG MARKER: == sanity test 19a: touch .../f19 ; ls -l ... ; rm .../f19 ===================================================================== 23:46:16 (1734392776)
[  915.620091] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19b: ls -l ...\/f19 \(should return error\) ======================================================================== 23:46:19 \(1734392779\)
[  915.981817] Lustre: DEBUG MARKER: == sanity test 19b: ls -l .../f19 (should return error) ======================================================================== 23:46:19 (1734392779)
[  922.694277] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19c: runas -u 500 -g 500 touch ...\/f19 \(should return error\) ============================================================ 23:46:26 \(1734392786\)
[  923.051322] Lustre: DEBUG MARKER: == sanity test 19c: runas -u 500 -g 500 touch .../f19 (should return error) ============================================================ 23:46:26 (1734392786)
[  924.773206] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 19d: cat ...\/f19 \(should return error\) ======================================================================== 23:46:28 \(1734392788\)
[  925.127650] Lustre: DEBUG MARKER: == sanity test 19d: cat .../f19 (should return error) ======================================================================== 23:46:28 (1734392788)
[  926.883585] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 20: touch ...\/f \; ls -l ... =============== 23:46:30 \(1734392790\)
[  927.241499] Lustre: DEBUG MARKER: == sanity test 20: touch .../f ; ls -l ... =============== 23:46:30 (1734392790)
[  928.996300] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 21: write to dangling link ================ 23:46:32 \(1734392792\)
[  929.357293] Lustre: DEBUG MARKER: == sanity test 21: write to dangling link ================ 23:46:32 (1734392792)
[  931.110095] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 22: unpack tar archive as non-root user === 23:46:34 \(1734392794\)
[  931.466766] Lustre: DEBUG MARKER: == sanity test 22: unpack tar archive as non-root user === 23:46:34 (1734392794)
[  933.247066] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23a: O_CREAT\|O_EXCL in subdir ============= 23:46:36 \(1734392796\)
[  933.605406] Lustre: DEBUG MARKER: == sanity test 23a: O_CREAT|O_EXCL in subdir ============= 23:46:36 (1734392796)
[  935.347081] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23b: O_APPEND check ======================= 23:46:38 \(1734392798\)
[  935.705548] Lustre: DEBUG MARKER: == sanity test 23b: O_APPEND check ======================= 23:46:38 (1734392798)
[  937.458890] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23c: O_APPEND size checks for tiny writes ========================================================== 23:46:40 \(1734392800\)
[  937.815382] Lustre: DEBUG MARKER: == sanity test 23c: O_APPEND size checks for tiny writes ========================================================== 23:46:40 (1734392800)
[  940.495507] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 23d: file offset is correct after appending writes ========================================================== 23:46:43 \(1734392803\)
[  940.850985] Lustre: DEBUG MARKER: == sanity test 23d: file offset is correct after appending writes ========================================================== 23:46:43 (1734392803)
[  942.590435] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24a: rename file to non-existent target === 23:46:46 \(1734392806\)
[  942.949580] Lustre: DEBUG MARKER: == sanity test 24a: rename file to non-existent target === 23:46:46 (1734392806)
[  944.716217] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24b: rename file to existing target ======= 23:46:48 \(1734392808\)
[  945.072990] Lustre: DEBUG MARKER: == sanity test 24b: rename file to existing target ======= 23:46:48 (1734392808)
[  946.862040] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24c: rename directory to non-existent target ========================================================== 23:46:50 \(1734392810\)
[  947.219504] Lustre: DEBUG MARKER: == sanity test 24c: rename directory to non-existent target ========================================================== 23:46:50 (1734392810)
[  948.995814] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24d: rename directory to existing target == 23:46:52 \(1734392812\)
[  949.352283] Lustre: DEBUG MARKER: == sanity test 24d: rename directory to existing target == 23:46:52 (1734392812)
[  951.105285] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24e: touch ...\/R5a\/f\; rename ...\/R5a\/f ...\/R5b\/g ================================================================ 23:46:54 \(1734392814\)
[  951.460847] Lustre: DEBUG MARKER: == sanity test 24e: touch .../R5a/f; rename .../R5a/f .../R5b/g ================================================================ 23:46:54 (1734392814)
[  953.212340] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24f: touch ...\/R6a\/f R6b\/g\; mv ...\/R6a\/f ...\/R6b\/g ============================================================== 23:46:56 \(1734392816\)
[  953.569568] Lustre: DEBUG MARKER: == sanity test 24f: touch .../R6a/f R6b/g; mv .../R6a/f .../R6b/g ============================================================== 23:46:56 (1734392816)
[  955.337333] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24g: mkdir ...\/R7{a,b}\/d\; mv ...\/R7a\/d ...\/R7b\/e ================================================================ 23:46:58 \(1734392818\)
[  955.697036] Lustre: DEBUG MARKER: == sanity test 24g: mkdir .../R7a/d; .../R7b/d; mv .../R7a/d .../R7b/e ================================================================ 23:46:58 (1734392818)
[  957.468324] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24h: mkdir ...\/R8{a,b}\/{d,e}\; rename ...\/R8a\/d ...\/R8b\/e ========================================================== 23:47:00 \(1734392820\)
[  957.825888] Lustre: DEBUG MARKER: == sanity test 24h: mkdir .../R8a/d; .../R8a/e; .../R8b/d; .../R8b/e; rename .../R8a/d .../R8b/e ========================================================== 23:47:00 (1734392820)
[  959.612587] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24i: rename file to dir error: touch f \; mkdir a \; rename f a ========================================================== 23:47:03 \(1734392823\)
[  959.972097] Lustre: DEBUG MARKER: == sanity test 24i: rename file to dir error: touch f ; mkdir a ; rename f a ========================================================== 23:47:03 (1734392823)
[  961.723186] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24j: source does not exist ====================================================================================== 23:47:05 \(1734392825\)
[  962.080960] Lustre: DEBUG MARKER: == sanity test 24j: source does not exist ====================================================================================== 23:47:05 (1734392825)
[  963.818762] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24k: touch ...\/R11a\/f\; mv ...\/R11a\/f ...\/R11a\/d ================================================================= 23:47:07 \(1734392827\)
[  964.174977] Lustre: DEBUG MARKER: == sanity test 24k: touch .../R11a/f; mv .../R11a/f .../R11a/d ================================================================= 23:47:07 (1734392827)
[  965.938655] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24l: Renaming a file to itself ================================================================================== 23:47:09 \(1734392829\)
[  966.292395] Lustre: DEBUG MARKER: == sanity test 24l: Renaming a file to itself ================================================================================== 23:47:09 (1734392829)
[  968.021057] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24m: Renaming a file to a hard link to itself =================================================================== 23:47:11 \(1734392831\)
[  968.381382] Lustre: DEBUG MARKER: == sanity test 24m: Renaming a file to a hard link to itself =================================================================== 23:47:11 (1734392831)
[  970.116922] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24n: Statting the old file after renaming \(Posix rename 2\) ========================================================== 23:47:13 \(1734392833\)
[  970.472427] Lustre: DEBUG MARKER: == sanity test 24n: Statting the old file after renaming (Posix rename 2) ========================================================== 23:47:13 (1734392833)
[  972.214100] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24o: rename of files during htree split === 23:47:15 \(1734392835\)
[  972.571854] Lustre: DEBUG MARKER: == sanity test 24o: rename of files during htree split === 23:47:15 (1734392835)
[ 1040.845314] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24p: mkdir ...\/R12{a,b}\; rename ...\/R12a ...\/R12b ========================================================== 23:48:24 \(1734392904\)
[ 1041.210835] Lustre: DEBUG MARKER: == sanity test 24p: mkdir .../R12a; .../R12b; rename .../R12a .../R12b ========================================================== 23:48:24 (1734392904)
[ 1052.979229] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24q: mkdir ...\/R13{a,b}\; open R13b rename R13a R13b ============================================================= 23:48:31 \(1734392911\)
[ 1053.338981] Lustre: DEBUG MARKER: == sanity test 24q: mkdir .../R13a; .../R13b; open R13b rename R13a R13b ============================================================= 23:48:31 (1734392911)
[ 1055.153093] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24r: mkdir ...\/R14a\/b\; rename ...\/R14a ...\/R14a\/b =============================================================== 23:48:38 \(1734392918\)
[ 1055.517125] Lustre: DEBUG MARKER: == sanity test 24r: mkdir .../R14a/b; rename .../R14a .../R14a/b =============================================================== 23:48:38 (1734392918)
[ 1057.283375] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24s: mkdir ...\/R15a\/b\/c\; rename ...\/R15a ...\/R15a\/b\/c =========================================================== 23:48:40 \(1734392920\)
[ 1057.644659] Lustre: DEBUG MARKER: == sanity test 24s: mkdir .../R15a/b/c; rename .../R15a .../R15a/b/c =========================================================== 23:48:40 (1734392920)
[ 1059.414255] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24t: mkdir ...\/R16a\/b\/c\; rename ...\/R16a\/b\/c ...\/R16a =========================================================== 23:48:42 \(1734392922\)
[ 1059.782977] Lustre: DEBUG MARKER: == sanity test 24t: mkdir .../R16a/b/c; rename .../R16a/b/c .../R16a =========================================================== 23:48:42 (1734392922)
[ 1061.557053] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24u: create stripe file =================== 23:48:44 \(1734392924\)
[ 1061.913152] Lustre: DEBUG MARKER: == sanity test 24u: create stripe file =================== 23:48:44 (1734392924)
[ 1064.108443] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24v: list large directory \(test hash collision, b=17560\) ========================================================== 23:48:47 \(1734392927\)
[ 1064.468802] Lustre: DEBUG MARKER: == sanity test 24v: list large directory (test hash collision, b=17560) ========================================================== 23:48:47 (1734392927)
[ 1065.179851] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1078.104984] Autotest: Test running for 15 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1089.013491] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1093.156792] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1133.661455] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1137.061065] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24w: Reading a file larger than 4Gb ======= 23:50:00 \(1734393000\)
[ 1137.420521] Lustre: DEBUG MARKER: == sanity test 24w: Reading a file larger than 4Gb ======= 23:50:00 (1734393000)
[ 1139.206598] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24x: cross MDT rename\/link ================ 23:50:02 \(1734393002\)
[ 1139.564506] Lustre: DEBUG MARKER: == sanity test 24x: cross MDT rename/link ================ 23:50:02 (1734393002)
[ 1139.970292] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24x needs \>= 2 MDTs
[ 1140.328476] Lustre: DEBUG MARKER: SKIP: sanity test_24x needs >= 2 MDTs
[ 1140.760070] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24y: rename\/link on the same dir should succeed ========================================================== 23:50:04 \(1734393004\)
[ 1141.117709] Lustre: DEBUG MARKER: == sanity test 24y: rename/link on the same dir should succeed ========================================================== 23:50:04 (1734393004)
[ 1141.516445] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24y needs \>= 2 MDTs
[ 1141.873073] Lustre: DEBUG MARKER: SKIP: sanity test_24y needs >= 2 MDTs
[ 1142.300518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24z: cross-MDT rename is done as cp ======= 23:50:05 \(1734393005\)
[ 1147.659194] Lustre: DEBUG MARKER: == sanity test 24z: cross-MDT rename is done as cp ======= 23:50:05 (1734393005)
[ 1148.059167] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24z needs \>= 2 MDTs
[ 1148.418006] Lustre: DEBUG MARKER: SKIP: sanity test_24z needs >= 2 MDTs
[ 1148.849019] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24A: readdir\(\) returns correct number of entries. ========================================================== 23:50:12 \(1734393012\)
[ 1149.212582] Lustre: DEBUG MARKER: == sanity test 24A: readdir() returns correct number of entries. ========================================================== 23:50:12 (1734393012)
[ 1149.881693] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1151.806008] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1157.091727] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1159.018456] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1162.057519] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24B: readdir for striped dir return correct number of entries ========================================================== 23:50:25 \(1734393025\)
[ 1162.416971] Lustre: DEBUG MARKER: == sanity test 24B: readdir for striped dir return correct number of entries ========================================================== 23:50:25 (1734393025)
[ 1162.819198] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24B needs \>= 2 MDTs
[ 1163.173856] Lustre: DEBUG MARKER: SKIP: sanity test_24B needs >= 2 MDTs
[ 1163.599385] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24C: check .. in striped dir ============== 23:50:27 \(1734393027\)
[ 1163.955250] Lustre: DEBUG MARKER: == sanity test 24C: check .. in striped dir ============== 23:50:27 (1734393027)
[ 1164.357223] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24C needs \>= 2 MDTs
[ 1164.710033] Lustre: DEBUG MARKER: SKIP: sanity test_24C needs >= 2 MDTs
[ 1165.138123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24E: cross MDT rename\/link ================ 23:50:28 \(1734393028\)
[ 1165.495831] Lustre: DEBUG MARKER: == sanity test 24E: cross MDT rename/link ================ 23:50:28 (1734393028)
[ 1165.895018] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24E needs \>= 4 MDTs
[ 1166.250655] Lustre: DEBUG MARKER: SKIP: sanity test_24E needs >= 4 MDTs
[ 1166.685660] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24F: hash order vs readdir \(LU-11330\) ===== 23:50:30 \(1734393030\)
[ 1167.042025] Lustre: DEBUG MARKER: == sanity test 24F: hash order vs readdir (LU-11330) ===== 23:50:30 (1734393030)
[ 1167.437460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24F needs \>= 2 MDTs
[ 1167.790241] Lustre: DEBUG MARKER: SKIP: sanity test_24F needs >= 2 MDTs
[ 1168.214896] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24G: migrate symlink in rename ============ 23:50:31 \(1734393031\)
[ 1168.572654] Lustre: DEBUG MARKER: == sanity test 24G: migrate symlink in rename ============ 23:50:31 (1734393031)
[ 1168.970511] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24G needs \>= 2 MDTs
[ 1169.322391] Lustre: DEBUG MARKER: SKIP: sanity test_24G needs >= 2 MDTs
[ 1169.750113] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 24H: repeat FLD_QUERY rpc ================= 23:50:33 \(1734393033\)
[ 1170.108503] Lustre: DEBUG MARKER: == sanity test 24H: repeat FLD_QUERY rpc ================= 23:50:33 (1734393033)
[ 1170.509510] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_24H needs \>= 2 MDTs
[ 1175.869061] Lustre: DEBUG MARKER: SKIP: sanity test_24H needs >= 2 MDTs
[ 1176.307368] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 25a: create file in symlinked directory ========================================================================= 23:50:39 \(1734393039\)
[ 1176.665734] Lustre: DEBUG MARKER: == sanity test 25a: create file in symlinked directory ========================================================================= 23:50:39 (1734393039)
[ 1178.417840] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 25b: lookup file in symlinked directory ========================================================================= 23:50:41 \(1734393041\)
[ 1178.777187] Lustre: DEBUG MARKER: == sanity test 25b: lookup file in symlinked directory ========================================================================= 23:50:41 (1734393041)
[ 1180.513879] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26a: multiple component symlink ================================================================================= 23:50:43 \(1734393043\)
[ 1180.880701] Lustre: DEBUG MARKER: == sanity test 26a: multiple component symlink ================================================================================= 23:50:43 (1734393043)
[ 1182.652913] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26b: multiple component symlink at end of lookup ================================================================ 23:50:46 \(1734393046\)
[ 1183.016034] Lustre: DEBUG MARKER: == sanity test 26b: multiple component symlink at end of lookup ================================================================ 23:50:46 (1734393046)
[ 1184.773894] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26c: chain of symlinks ==================== 23:50:48 \(1734393048\)
[ 1185.143559] Lustre: DEBUG MARKER: == sanity test 26c: chain of symlinks ==================== 23:50:48 (1734393048)
[ 1191.922189] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26d: create multiple component recursive symlink ========================================================== 23:50:50 \(1734393050\)
[ 1192.283039] Lustre: DEBUG MARKER: == sanity test 26d: create multiple component recursive symlink ========================================================== 23:50:50 (1734393050)
[ 1194.035874] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26e: unlink multiple component recursive symlink ========================================================== 23:50:57 \(1734393057\)
[ 1194.401299] Lustre: DEBUG MARKER: == sanity test 26e: unlink multiple component recursive symlink ========================================================== 23:50:57 (1734393057)
[ 1196.145084] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 26f: rm -r of a directory which has recursive symlink ========================================================== 23:50:59 \(1734393059\)
[ 1196.501825] Lustre: DEBUG MARKER: == sanity test 26f: rm -r of a directory which has recursive symlink ========================================================== 23:50:59 (1734393059)
[ 1198.413981] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27a: one stripe file ====================== 23:51:01 \(1734393061\)
[ 1198.777729] Lustre: DEBUG MARKER: == sanity test 27a: one stripe file ====================== 23:51:01 (1734393061)
[ 1200.565452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27b: create and write to two stripe file == 23:51:04 \(1734393064\)
[ 1200.927347] Lustre: DEBUG MARKER: == sanity test 27b: create and write to two stripe file == 23:51:04 (1734393064)
[ 1202.711877] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27ca: one stripe on specified OST ========= 23:51:06 \(1734393066\)
[ 1208.072301] Lustre: DEBUG MARKER: == sanity test 27ca: one stripe on specified OST ========= 23:51:06 (1734393066)
[ 1209.977394] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cb: two stripes on specified OSTs ======= 23:51:13 \(1734393073\)
[ 1210.339278] Lustre: DEBUG MARKER: == sanity test 27cb: two stripes on specified OSTs ======= 23:51:13 (1734393073)
[ 1212.323909] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cc: two stripes on the same OST ========= 23:51:15 \(1734393075\)
[ 1212.683286] Lustre: DEBUG MARKER: == sanity test 27cc: two stripes on the same OST ========= 23:51:15 (1734393075)
[ 1214.661510] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cd: four stripes on two OSTs ============ 23:51:18 \(1734393078\)
[ 1215.019948] Lustre: DEBUG MARKER: == sanity test 27cd: four stripes on two OSTs ============ 23:51:18 (1734393078)
[ 1216.907691] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27ce: more stripes than OSTs with -o ====== 23:51:20 \(1734393080\)
[ 1217.268613] Lustre: DEBUG MARKER: == sanity test 27ce: more stripes than OSTs with -o ====== 23:51:20 (1734393080)
[ 1219.171370] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cf: \'setstripe -o\' on inactive OSTs should return error ========================================================== 23:51:22 \(1734393082\)
[ 1219.532495] Lustre: DEBUG MARKER: == sanity test 27cf: 'setstripe -o' on inactive OSTs should return error ========================================================== 23:51:22 (1734393082)
[ 1223.878318] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27cg: 1000 shouldn\'t cause too many credits ========================================================== 23:51:27 \(1734393087\)
[ 1224.237837] Lustre: DEBUG MARKER: == sanity test 27cg: 1000 shouldn't cause too many credits ========================================================== 23:51:27 (1734393087)
[ 1227.529838] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27d: create file with default settings ==== 23:51:30 \(1734393090\)
[ 1227.889070] Lustre: DEBUG MARKER: == sanity test 27d: create file with default settings ==== 23:51:30 (1734393090)
[ 1229.697251] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27e: setstripe existing file \(should return error\) ========================================================== 23:51:33 \(1734393093\)
[ 1230.059118] Lustre: DEBUG MARKER: == sanity test 27e: setstripe existing file (should return error) ========================================================== 23:51:33 (1734393093)
[ 1231.865444] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27f: setstripe with bad stripe size \(should return error\) ========================================================== 23:51:35 \(1734393095\)
[ 1232.322378] Lustre: DEBUG MARKER: == sanity test 27f: setstripe with bad stripe size (should return error) ========================================================== 23:51:35 (1734393095)
[ 1234.189620] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27g: \/usr\/bin\/lfs getstripe with no objects ========================================================== 23:51:37 \(1734393097\)
[ 1234.560746] Lustre: DEBUG MARKER: == sanity test 27g: /usr/bin/lfs getstripe with no objects ========================================================== 23:51:37 (1734393097)
[ 1236.327591] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27ga: \/usr\/bin\/lfs getstripe with missing file \(should return error\) ========================================================== 23:51:39 \(1734393099\)
[ 1236.685323] Lustre: DEBUG MARKER: == sanity test 27ga: /usr/bin/lfs getstripe with missing file (should return error) ========================================================== 23:51:39 (1734393099)
[ 1238.459456] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27i: \/usr\/bin\/lfs getstripe with some objects ========================================================== 23:51:41 \(1734393101\)
[ 1238.818424] Lustre: DEBUG MARKER: == sanity test 27i: /usr/bin/lfs getstripe with some objects ========================================================== 23:51:41 (1734393101)
[ 1240.589067] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27j: setstripe with bad stripe offset \(should return error\) ========================================================== 23:51:44 \(1734393104\)
[ 1240.945062] Lustre: DEBUG MARKER: == sanity test 27j: setstripe with bad stripe offset (should return error) ========================================================== 23:51:44 (1734393104)
[ 1242.681427] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27k: limit i_blksize for broken user apps ========================================================== 23:51:46 \(1734393106\)
[ 1243.035911] Lustre: DEBUG MARKER: == sanity test 27k: limit i_blksize for broken user apps ========================================================== 23:51:46 (1734393106)
[ 1244.801077] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27l: check setstripe permissions \(should return error\) ========================================================== 23:51:48 \(1734393108\)
[ 1245.158712] Lustre: DEBUG MARKER: == sanity test 27l: check setstripe permissions (should return error) ========================================================== 23:51:48 (1734393108)
[ 1246.878082] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_27m skipping SLOW test 27m
[ 1247.233229] Lustre: DEBUG MARKER: SKIP: sanity test_27m skipping SLOW test 27m
[ 1247.652654] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27n: create file with some full OSTs ====== 23:51:51 \(1734393111\)
[ 1248.009068] Lustre: DEBUG MARKER: == sanity test 27n: create file with some full OSTs ====== 23:51:51 (1734393111)
[ 1268.035367] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27o: create file with all full OSTs \(should error\) ========================================================== 23:52:11 \(1734393131\)
[ 1268.388926] Lustre: DEBUG MARKER: == sanity test 27o: create file with all full OSTs (should error) ========================================================== 23:52:11 (1734393131)
[ 1301.458601] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27oo: don\'t let few threads to reserve too many objects ========================================================== 23:52:44 \(1734393164\)
[ 1301.814088] Lustre: DEBUG MARKER: == sanity test 27oo: don't let few threads to reserve too many objects ========================================================== 23:52:44 (1734393164)
[ 1331.874233] Lustre: lustre-OST0000-osc-ffff988d17847800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1335.293078] Lustre: lustre-OST0000-osc-ffff988d17847800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[ 1337.893452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27p: append to a truncated file with some full OSTs ========================================================== 23:53:21 \(1734393201\)
[ 1338.253872] Lustre: DEBUG MARKER: == sanity test 27p: append to a truncated file with some full OSTs ========================================================== 23:53:21 (1734393201)
[ 1357.472246] Lustre: lustre-OST0000-osc-ffff988d17847800: disconnect after 23s idle
[ 1367.106926] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27q: append to truncated file with all OSTs full \(should error\) ========================================================== 23:53:50 \(1734393230\)
[ 1367.464969] Lustre: DEBUG MARKER: == sanity test 27q: append to truncated file with all OSTs full (should error) ========================================================== 23:53:50 (1734393230)
[ 1378.828750] Autotest: Test running for 20 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1400.407380] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27r: stripe file with some full OSTs \(shouldn\'t LBUG\) =========================================================== 23:54:23 \(1734393263\)
[ 1400.768491] Lustre: DEBUG MARKER: == sanity test 27r: stripe file with some full OSTs (shouldn't LBUG) =========================================================== 23:54:23 (1734393263)
[ 1420.822149] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27s: lsm_xfersize overflow \(should error\) \(bug 10725\) ========================================================== 23:54:44 \(1734393284\)
[ 1421.184154] Lustre: DEBUG MARKER: == sanity test 27s: lsm_xfersize overflow (should error) (bug 10725) ========================================================== 23:54:44 (1734393284)
[ 1422.931440] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27t: check that utils parse path correctly ========================================================== 23:54:46 \(1734393286\)
[ 1423.284247] Lustre: DEBUG MARKER: == sanity test 27t: check that utils parse path correctly ========================================================== 23:54:46 (1734393286)
[ 1425.062218] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27u: skip object creation on OSC w\/o objects ========================================================== 23:54:48 \(1734393288\)
[ 1425.422346] Lustre: DEBUG MARKER: == sanity test 27u: skip object creation on OSC w/o objects ========================================================== 23:54:48 (1734393288)
[ 1426.475134] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1427.711180] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1431.713281] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 1432.665092] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 1444.511823] Lustre: lustre-OST0006-osc-ffff988d17847800: disconnect after 22s idle
[ 1444.571851] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27v: skip object creation on slow OST ===== 23:55:08 \(1734393308\)
[ 1445.021764] Lustre: DEBUG MARKER: == sanity test 27v: skip object creation on slow OST ===== 23:55:08 (1734393308)
[ 1454.751658] Lustre: lustre-OST0001-osc-ffff988d17847800: disconnect after 22s idle
[ 1470.111613] Lustre: lustre-OST0005-osc-ffff988d17847800: disconnect after 20s idle
[ 1470.113166] Lustre: Skipped 1 previous similar message
[ 1491.802179] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27w: check \/usr\/bin\/lfs setstripe -S and getstrip -d options ========================================================== 23:55:55 \(1734393355\)
[ 1492.160519] Lustre: DEBUG MARKER: == sanity test 27w: check /usr/bin/lfs setstripe -S and getstrip -d options ========================================================== 23:55:55 (1734393355)
[ 1493.954581] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27wa: check \/usr\/bin\/lfs setstripe -c -i options ========================================================== 23:55:57 \(1734393357\)
[ 1494.313775] Lustre: DEBUG MARKER: == sanity test 27wa: check /usr/bin/lfs setstripe -c -i options ========================================================== 23:55:57 (1734393357)
[ 1495.711411] Lustre: lustre-OST0002-osc-ffff988d17847800: disconnect after 21s idle
[ 1495.712871] Lustre: Skipped 2 previous similar messages
[ 1496.211565] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27x: create files while OST0 is degraded == 23:55:59 \(1734393359\)
[ 1496.566205] Lustre: DEBUG MARKER: == sanity test 27x: create files while OST0 is degraded == 23:55:59 (1734393359)
[ 1509.552591] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27y: create files while OST0 is degraded and the rest inactive ========================================================== 23:56:12 \(1734393372\)
[ 1509.910005] Lustre: DEBUG MARKER: == sanity test 27y: create files while OST0 is degraded and the rest inactive ========================================================== 23:56:12 (1734393372)
[ 1542.993633] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27z: check SEQ\/OID on the MDT and OST filesystems ========================================================== 23:56:46 \(1734393406\)
[ 1543.354345] Lustre: DEBUG MARKER: == sanity test 27z: check SEQ/OID on the MDT and OST filesystems ========================================================== 23:56:46 (1734393406)
[ 1549.634068] Lustre: DEBUG MARKER: /usr/sbin/lctl mark check file \/mnt\/lustre\/d27z.sanity\/f27z.sanity-1
[ 1549.988221] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-1
[ 1550.410166] Lustre: DEBUG MARKER: /usr/sbin/lctl mark FID seq 0x200000401, oid 0x1cdba ver 0x0
[ 1550.762501] Lustre: DEBUG MARKER: FID seq 0x200000401, oid 0x1cdba ver 0x0
[ 1551.162386] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LOV seq 0x200000401, oid 0x1cdba, count: 1
[ 1551.514955] Lustre: DEBUG MARKER: LOV seq 0x200000401, oid 0x1cdba, count: 1
[ 1551.920058] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:0 ost:0 oid:372\/0x174 seq:0x240000400
[ 1552.278793] Lustre: DEBUG MARKER: want: stripe:0 ost:0 oid:372/0x174 seq:0x240000400
[ 1553.080144] Lustre: DEBUG MARKER: /usr/sbin/lctl mark check file \/mnt\/lustre\/d27z.sanity\/f27z.sanity-2
[ 1553.440319] Lustre: DEBUG MARKER: check file /mnt/lustre/d27z.sanity/f27z.sanity-2
[ 1553.870759] Lustre: DEBUG MARKER: /usr/sbin/lctl mark FID seq 0x200000401, oid 0x1cdbc ver 0x0
[ 1554.227176] Lustre: DEBUG MARKER: FID seq 0x200000401, oid 0x1cdbc ver 0x0
[ 1554.634429] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LOV seq 0x200000401, oid 0x1cdbc, count: 7
[ 1554.990126] Lustre: DEBUG MARKER: LOV seq 0x200000401, oid 0x1cdbc, count: 7
[ 1555.411928] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:0 ost:6 oid:399\/0x18f seq:0x3c0000400
[ 1555.769276] Lustre: DEBUG MARKER: want: stripe:0 ost:6 oid:399/0x18f seq:0x3c0000400
[ 1556.542576] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:1 ost:0 oid:373\/0x175 seq:0x240000400
[ 1556.901728] Lustre: DEBUG MARKER: want: stripe:1 ost:0 oid:373/0x175 seq:0x240000400
[ 1557.716365] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:2 ost:1 oid:526\/0x20e seq:0x280000400
[ 1558.086268] Lustre: DEBUG MARKER: want: stripe:2 ost:1 oid:526/0x20e seq:0x280000400
[ 1558.857796] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:3 ost:2 oid:399\/0x18f seq:0x2c0000400
[ 1559.212143] Lustre: DEBUG MARKER: want: stripe:3 ost:2 oid:399/0x18f seq:0x2c0000400
[ 1560.021397] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:4 ost:3 oid:400\/0x190 seq:0x300000400
[ 1560.390395] Lustre: DEBUG MARKER: want: stripe:4 ost:3 oid:400/0x190 seq:0x300000400
[ 1561.231607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:5 ost:4 oid:400\/0x190 seq:0x340000400
[ 1561.588978] Lustre: DEBUG MARKER: want: stripe:5 ost:4 oid:400/0x190 seq:0x340000400
[ 1562.354323] Lustre: DEBUG MARKER: /usr/sbin/lctl mark want: stripe:6 ost:5 oid:400\/0x190 seq:0x380000400
[ 1562.708357] Lustre: DEBUG MARKER: want: stripe:6 ost:5 oid:400/0x190 seq:0x380000400
[ 1564.852572] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27A: check filesystem-wide default LOV EA values ========================================================== 23:57:08 \(1734393428\)
[ 1565.210192] Lustre: DEBUG MARKER: == sanity test 27A: check filesystem-wide default LOV EA values ========================================================== 23:57:08 (1734393428)
[ 1565.237948] Lustre: DEBUG MARKER: /usr/bin/lfs getstripe -c /mnt/lustre | sed 's/  *//g'
[ 1567.386580] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27B: call setstripe on open unlinked file\/rename victim ========================================================== 23:57:10 \(1734393430\)
[ 1567.744178] Lustre: DEBUG MARKER: == sanity test 27B: call setstripe on open unlinked file/rename victim ========================================================== 23:57:10 (1734393430)
[ 1569.524476] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ca: check full striping across all OSTs ========================================================== 23:57:12 \(1734393432\)
[ 1569.883273] Lustre: DEBUG MARKER: == sanity test 27Ca: check full striping across all OSTs ========================================================== 23:57:12 (1734393432)
[ 1571.880087] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cb: more stripes than OSTs with -C ====== 23:57:15 \(1734393435\)
[ 1572.236130] Lustre: DEBUG MARKER: == sanity test 27Cb: more stripes than OSTs with -C ====== 23:57:15 (1734393435)
[ 1574.117085] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cc: fewer stripes than OSTs does not set overstriping ========================================================== 23:57:17 \(1734393437\)
[ 1574.474331] Lustre: DEBUG MARKER: == sanity test 27Cc: fewer stripes than OSTs does not set overstriping ========================================================== 23:57:17 (1734393437)
[ 1576.333362] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cd: test maximum stripe count =========== 23:57:19 \(1734393439\)
[ 1576.692387] Lustre: DEBUG MARKER: == sanity test 27Cd: test maximum stripe count =========== 23:57:19 (1734393439)
[ 1595.877525] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ce: test pool with overstriping ========= 23:57:39 \(1734393459\)
[ 1596.235047] Lustre: DEBUG MARKER: == sanity test 27Ce: test pool with overstriping ========= 23:57:39 (1734393459)
[ 1598.579422] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ce 		2>/dev/null || echo foo
[ 1599.993196] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ce |
[ 1599.993196] 			grep -e lustre-OST0000_UUID  | sort -u | tr '\n' ' ' 
[ 1607.041271] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ce 		2>/dev/null || echo foo
[ 1607.479234] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cf: test default inheritance with overstriping ========================================================== 23:57:50 \(1734393470\)
[ 1612.841865] Lustre: DEBUG MARKER: == sanity test 27Cf: test default inheritance with overstriping ========================================================== 23:57:50 (1734393470)
[ 1614.755353] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cg: test setstripe with wrong OST idx === 23:57:58 \(1734393478\)
[ 1615.112694] Lustre: DEBUG MARKER: == sanity test 27Cg: test setstripe with wrong OST idx === 23:57:58 (1734393478)
[ 1616.865444] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ci: add an overstriping component ======= 23:58:00 \(1734393480\)
[ 1617.223661] Lustre: DEBUG MARKER: == sanity test 27Ci: add an overstriping component ======= 23:58:00 (1734393480)
[ 1619.362780] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Cj: overstriping with -C for max values in multiple of targets ========================================================== 23:58:02 \(1734393482\)
[ 1619.723379] Lustre: DEBUG MARKER: == sanity test 27Cj: overstriping with -C for max values in multiple of targets ========================================================== 23:58:02 (1734393482)
[ 1621.600098] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27D: validate llapi_layout API ============ 23:58:05 \(1734393485\)
[ 1621.964332] Lustre: DEBUG MARKER: == sanity test 27D: validate llapi_layout API ============ 23:58:05 (1734393485)
[ 1624.325819] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 		2>/dev/null || echo foo
[ 1626.676294] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool |
[ 1626.676294] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID  | sort -u | tr '\n' ' ' 
[ 1646.408466] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 		2>/dev/null || echo foo
[ 1648.232409] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27E: check that default extended attribute size properly increases ========================================================== 23:58:31 \(1734393511\)
[ 1648.588377] Lustre: DEBUG MARKER: == sanity test 27E: check that default extended attribute size properly increases ========================================================== 23:58:31 (1734393511)
[ 1655.364884] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27F: Client resend delayed layout creation with non-zero size ========================================================== 23:58:38 \(1734393518\)
[ 1655.719753] Lustre: DEBUG MARKER: == sanity test 27F: Client resend delayed layout creation with non-zero size ========================================================== 23:58:38 (1734393518)
[ 1659.552138] Lustre: lustre-OST0001-osc-ffff988d17847800: Connection to lustre-OST0001 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1659.555341] Lustre: Skipped 2 previous similar messages
[ 1664.672412] Lustre: lustre-OST0004-osc-ffff988d17847800: Connection to lustre-OST0004 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 1669.891251] Lustre: lustre-OST0000-osc-ffff988d17847800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[ 1679.838426] Autotest: Test running for 25 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1682.701253] Lustre: lustre-OST0002-osc-ffff988d17847800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[ 1682.703266] Lustre: Skipped 1 previous similar message
[ 1690.270261] Lustre: lustre-OST0000-osc-ffff988d17847800: disconnect after 21s idle
[ 1701.098173] Lustre: lustre-OST0005-osc-ffff988d17847800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[ 1701.102161] Lustre: Skipped 2 previous similar messages
[ 1711.151729] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27G: Clear OST pool from stripe =========== 23:59:34 \(1734393574\)
[ 1711.506705] Lustre: DEBUG MARKER: == sanity test 27G: Clear OST pool from stripe =========== 23:59:34 (1734393574)
[ 1713.936032] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 		2>/dev/null || echo foo
[ 1716.281798] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool |
[ 1716.281798] 			grep -e lustre-OST0000_UUID  | sort -u | tr '\n' ' ' 
[ 1723.261307] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 		2>/dev/null || echo foo
[ 1723.701265] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27H: Set specific OSTs stripe ============= 23:59:47 \(1734393587\)
[ 1724.058674] Lustre: DEBUG MARKER: == sanity test 27H: Set specific OSTs stripe ============= 23:59:47 (1734393587)
[ 1725.942293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27I: check that root dir striping does not break parent dir one ========================================================== 23:59:49 \(1734393589\)
[ 1726.301285] Lustre: DEBUG MARKER: == sanity test 27I: check that root dir striping does not break parent dir one ========================================================== 23:59:49 (1734393589)
[ 1728.640302] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27I 		2>/dev/null || echo foo
[ 1730.986290] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27I |
[ 1730.986290] 			grep -e lustre-OST0001_UUID  | sort -u | tr '\n' ' ' 
[ 1737.865049] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27I 		2>/dev/null || echo foo
[ 1738.337791] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Ia: check that root dir pool is dropped with conflict parent dir settings ========================================================== 00:00:01 \(1734393601\)
[ 1738.696632] Lustre: DEBUG MARKER: == sanity test 27Ia: check that root dir pool is dropped with conflict parent dir settings ========================================================== 00:00:01 (1734393601)
[ 1741.036305] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ia 		2>/dev/null || echo foo
[ 1743.381989] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ia |
[ 1743.381989] 			grep -e lustre-OST0001_UUID  | sort -u | tr '\n' ' ' 
[ 1750.402407] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27Ia 		2>/dev/null || echo foo
[ 1750.840651] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27J: basic ops on file with foreign LOV === 00:00:14 \(1734393614\)
[ 1751.193818] Lustre: DEBUG MARKER: == sanity test 27J: basic ops on file with foreign LOV === 00:00:14 (1734393614)
[ 1753.096613] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27K: basic ops on dir with foreign LMV ==== 00:00:16 \(1734393616\)
[ 1753.453366] Lustre: DEBUG MARKER: == sanity test 27K: basic ops on dir with foreign LMV ==== 00:00:16 (1734393616)
[ 1755.351713] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27L: lfs pool_list gives correct pool name ========================================================== 00:00:18 \(1734393618\)
[ 1755.710560] Lustre: DEBUG MARKER: == sanity test 27L: lfs pool_list gives correct pool name ========================================================== 00:00:18 (1734393618)
[ 1758.063065] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27L 		2>/dev/null || echo foo
[ 1762.657079] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27L 		2>/dev/null || echo foo
[ 1763.089936] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27M: test O_APPEND striping =============== 00:00:26 \(1734393626\)
[ 1763.450199] Lustre: DEBUG MARKER: == sanity test 27M: test O_APPEND striping =============== 00:00:26 (1734393626)
[ 1768.943086] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27M 		2>/dev/null || echo foo
[ 1771.292967] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27M |
[ 1771.292967] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID  | sort -u | tr '\n' ' ' 
[ 1781.554594] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27M 		2>/dev/null || echo foo
[ 1783.222775] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27N: lctl pool_list on separate MGS gives correct pool name ========================================================== 00:00:46 \(1734393646\)
[ 1783.579905] Lustre: DEBUG MARKER: == sanity test 27N: lctl pool_list on separate MGS gives correct pool name ========================================================== 00:00:46 (1734393646)
[ 1784.000907] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_27N needs separate MGS\/MDT
[ 1784.359406] Lustre: DEBUG MARKER: SKIP: sanity test_27N needs separate MGS/MDT
[ 1784.789884] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27O: basic ops on foreign file of symlink type ========================================================== 00:00:48 \(1734393648\)
[ 1785.153170] Lustre: DEBUG MARKER: == sanity test 27O: basic ops on foreign file of symlink type ========================================================== 00:00:48 (1734393648)
[ 1792.411657] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27P: basic ops on foreign dir of foreign_symlink type ========================================================== 00:00:55 \(1734393655\)
[ 1797.777129] Lustre: DEBUG MARKER: == sanity test 27P: basic ops on foreign dir of foreign_symlink type ========================================================== 00:00:55 (1734393655)
[ 1800.004628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27Q: llapi_file_get_stripe\(\) works on symlinks ========================================================== 00:01:03 \(1734393663\)
[ 1800.360535] Lustre: DEBUG MARKER: == sanity test 27Q: llapi_file_get_stripe() works on symlinks ========================================================== 00:01:03 (1734393663)
[ 1802.167871] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27R: test max_stripecount limitation when stripe count is set to -1 ========================================================== 00:01:05 \(1734393665\)
[ 1802.531570] Lustre: DEBUG MARKER: == sanity test 27R: test max_stripecount limitation when stripe count is set to -1 ========================================================== 00:01:05 (1734393665)
[ 1805.334595] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27T: no eio on close on partial write due to enosp ========================================================== 00:01:08 \(1734393668\)
[ 1805.691002] Lustre: DEBUG MARKER: == sanity test 27T: no eio on close on partial write due to enosp ========================================================== 00:01:08 (1734393668)
[ 1807.841161] Lustre: *** cfs_fail_loc=411, val=1***
[ 1810.213275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27U: append pool and stripe count work with composite default layout ========================================================== 00:01:13 \(1734393673\)
[ 1810.565999] Lustre: DEBUG MARKER: == sanity test 27U: append pool and stripe count work with composite default layout ========================================================== 00:01:13 (1734393673)
[ 1813.515791] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-append 		2>/dev/null || echo foo
[ 1815.857844] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-append |
[ 1815.857844] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID  | sort -u | tr '\n' ' ' 
[ 1817.183359] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-normal 		2>/dev/null || echo foo
[ 1819.526198] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-normal |
[ 1819.526198] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID  | sort -u | tr '\n' ' ' 
[ 1823.389519] Lustre: lustre-OST0003-osc-ffff988d17847800: disconnect after 21s idle
[ 1823.391150] Lustre: Skipped 7 previous similar messages
[ 1829.470274] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-append 		2>/dev/null || echo foo
[ 1837.381107] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_27U-normal 		2>/dev/null || echo foo
[ 1838.502427] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27V: creating widely striped file races with deactivating OST ========================================================== 00:01:41 \(1734393701\)
[ 1838.949714] Lustre: DEBUG MARKER: == sanity test 27V: creating widely striped file races with deactivating OST ========================================================== 00:01:41 (1734393701)
[ 1844.164593] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 27W: test enable_setstripe_gid ============ 00:01:47 \(1734393707\)
[ 1844.528774] Lustre: DEBUG MARKER: == sanity test 27W: test enable_setstripe_gid ============ 00:01:47 (1734393707)
[ 1846.397634] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 28: create\/mknod\/mkdir with bad file types ====================================================================== 00:01:49 \(1734393709\)
[ 1846.751544] Lustre: DEBUG MARKER: == sanity test 28: create/mknod/mkdir with bad file types ====================================================================== 00:01:49 (1734393709)
[ 1848.659490] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 29: IT_GETATTR regression  ====================================================================================== 00:01:52 \(1734393712\)
[ 1849.016275] Lustre: DEBUG MARKER: == sanity test 29: IT_GETATTR regression ====================================================================================== 00:01:52 (1734393712)
[ 1850.954717] Lustre: DEBUG MARKER: /usr/sbin/lctl mark first d29
[ 1851.311385] Lustre: DEBUG MARKER: first d29
[ 1851.745785] Lustre: DEBUG MARKER: /usr/sbin/lctl mark second d29
[ 1852.105825] Lustre: DEBUG MARKER: second d29
[ 1852.517011] Lustre: DEBUG MARKER: /usr/sbin/lctl mark done
[ 1852.875309] Lustre: DEBUG MARKER: done
[ 1854.663177] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30a: execute binary from Lustre \(execve\) ======================================================================== 00:01:58 \(1734393718\)
[ 1855.022903] Lustre: DEBUG MARKER: == sanity test 30a: execute binary from Lustre (execve) ======================================================================== 00:01:58 (1734393718)
[ 1856.782516] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30b: execute binary from Lustre as non-root ===================================================================== 00:02:00 \(1734393720\)
[ 1857.136999] Lustre: DEBUG MARKER: == sanity test 30b: execute binary from Lustre as non-root ===================================================================== 00:02:00 (1734393720)
[ 1858.916174] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30c: execute binary from Lustre without read perms ============================================================== 00:02:02 \(1734393722\)
[ 1859.274127] Lustre: DEBUG MARKER: == sanity test 30c: execute binary from Lustre without read perms ============================================================== 00:02:02 (1734393722)
[ 1861.060476] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 30d: execute binary from Lustre while clear locks ========================================================== 00:02:04 \(1734393724\)
[ 1861.423648] Lustre: DEBUG MARKER: == sanity test 30d: execute binary from Lustre while clear locks ========================================================== 00:02:04 (1734393724)
[ 1904.034271] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31a: open-unlink file ============================================================================================ 00:02:47 \(1734393767\)
[ 1904.396106] Lustre: DEBUG MARKER: == sanity test 31a: open-unlink file ============================================================================================ 00:02:47 (1734393767)
[ 1906.216920] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31b: unlink file with multiple links while open ================================================================= 00:02:49 \(1734393769\)
[ 1906.572237] Lustre: DEBUG MARKER: == sanity test 31b: unlink file with multiple links while open ================================================================= 00:02:49 (1734393769)
[ 1908.361529] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31c: open-unlink file with multiple links ======================================================================= 00:02:51 \(1734393771\)
[ 1908.725204] Lustre: DEBUG MARKER: == sanity test 31c: open-unlink file with multiple links ======================================================================= 00:02:51 (1734393771)
[ 1910.521538] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31d: remove of open directory =================================================================================== 00:02:53 \(1734393773\)
[ 1910.879320] Lustre: DEBUG MARKER: == sanity test 31d: remove of open directory =================================================================================== 00:02:53 (1734393773)
[ 1912.635890] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31e: remove of open non-empty directory ========================================================================= 00:02:56 \(1734393776\)
[ 1912.997706] Lustre: DEBUG MARKER: == sanity test 31e: remove of open non-empty directory ========================================================================= 00:02:56 (1734393776)
[ 1914.786240] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31f: remove of open directory with open-unlink file ============================================================= 00:02:58 \(1734393778\)
[ 1915.150871] Lustre: DEBUG MARKER: == sanity test 31f: remove of open directory with open-unlink file ============================================================= 00:02:58 (1734393778)
[ 1928.128328] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31g: cross directory link================== 00:03:11 \(1734393791\)
[ 1928.487037] Lustre: DEBUG MARKER: == sanity test 31g: cross directory link================== 00:03:11 (1734393791)
[ 1930.381580] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31h: cross directory link under child========================================================================= 00:03:13 \(1734393793\)
[ 1930.741634] Lustre: DEBUG MARKER: == sanity test 31h: cross directory link under child========================================================================= 00:03:13 (1734393793)
[ 1932.559995] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31i: cross directory link under parent========================================================================= 00:03:16 \(1734393796\)
[ 1932.919809] Lustre: DEBUG MARKER: == sanity test 31i: cross directory link under parent========================================================================= 00:03:16 (1734393796)
[ 1934.747471] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31j: link for directory =================== 00:03:18 \(1734393798\)
[ 1935.102093] Lustre: DEBUG MARKER: == sanity test 31j: link for directory =================== 00:03:18 (1734393798)
[ 1936.889399] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31k: link to file: the same, non-existing, dir ========================================================== 00:03:20 \(1734393800\)
[ 1937.244931] Lustre: DEBUG MARKER: == sanity test 31k: link to file: the same, non-existing, dir ========================================================== 00:03:20 (1734393800)
[ 1939.193107] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31l: link to file: target dir has trailing slash ========================================================== 00:03:22 \(1734393802\)
[ 1939.552269] Lustre: DEBUG MARKER: == sanity test 31l: link to file: target dir has trailing slash ========================================================== 00:03:22 (1734393802)
[ 1939.975372] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_31l need coreutils \< 8.31 or kernel \>= 5.18 for ln
[ 1940.336132] Lustre: DEBUG MARKER: SKIP: sanity test_31l need coreutils < 8.31 or kernel >= 5.18 for ln
[ 1940.766627] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31m: link to file: the same, non-existing, dir ========================================================== 00:03:24 \(1734393804\)
[ 1941.128759] Lustre: DEBUG MARKER: == sanity test 31m: link to file: the same, non-existing, dir ========================================================== 00:03:24 (1734393804)
[ 1942.912304] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31n: check link count of unlinked file ==== 00:03:26 \(1734393806\)
[ 1943.272726] Lustre: DEBUG MARKER: == sanity test 31n: check link count of unlinked file ==== 00:03:26 (1734393806)
[ 1950.074994] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31o: duplicate hard links with same filename ========================================================== 00:03:33 \(1734393813\)
[ 1950.438463] Lustre: DEBUG MARKER: == sanity test 31o: duplicate hard links with same filename ========================================================== 00:03:33 (1734393813)
[ 1959.361029] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31p: remove of open striped directory ===== 00:03:42 \(1734393822\)
[ 1959.725883] Lustre: DEBUG MARKER: == sanity test 31p: remove of open striped directory ===== 00:03:42 (1734393822)
[ 1960.124783] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_31p needs \>= 2 MDTs
[ 1960.484660] Lustre: DEBUG MARKER: SKIP: sanity test_31p needs >= 2 MDTs
[ 1960.927031] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31q: create striped directory on specific MDTs ========================================================== 00:03:44 \(1734393824\)
[ 1961.284786] Lustre: DEBUG MARKER: == sanity test 31q: create striped directory on specific MDTs ========================================================== 00:03:44 (1734393824)
[ 1961.702366] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_31q needs \>= 3 MDTs
[ 1962.055958] Lustre: DEBUG MARKER: SKIP: sanity test_31q needs >= 3 MDTs
[ 1962.484451] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 31r: open-rename\(replace\) race ============ 00:03:45 \(1734393825\)
[ 1962.837815] Lustre: DEBUG MARKER: == sanity test 31r: open-rename(replace) race ============ 00:03:45 (1734393825)
[ 1965.665135] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32a: stat d32a\/ext2-mountpoint\/.. =============================================================================== 00:03:49 \(1734393829\)
[ 1966.028558] Lustre: DEBUG MARKER: == sanity test 32a: stat d32a/ext2-mountpoint/.. =============================================================================== 00:03:49 (1734393829)
[ 1966.058418] loop0: detected capacity change from 0 to 16000
[ 1966.059925] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1966.738816] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1968.672554] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32b: open d32b\/ext2-mountpoint\/.. =============================================================================== 00:03:52 \(1734393832\)
[ 1969.095209] Lustre: DEBUG MARKER: == sanity test 32b: open d32b/ext2-mountpoint/.. =============================================================================== 00:03:52 (1734393832)
[ 1969.126514] loop0: detected capacity change from 0 to 16000
[ 1969.127932] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1969.322502] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1971.208979] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32c: stat d32c\/ext2-mountpoint\/..\/d2\/test_dir =================================================================== 00:03:54 \(1734393834\)
[ 1971.569843] Lustre: DEBUG MARKER: == sanity test 32c: stat d32c/ext2-mountpoint/../d2/test_dir =================================================================== 00:03:54 (1734393834)
[ 1971.603175] loop0: detected capacity change from 0 to 16000
[ 1971.604733] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1971.629539] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1973.506646] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32d: open d32d\/ext2-mountpoint\/..\/d2\/test_dir ========================================================== 00:03:56 \(1734393836\)
[ 1973.867421] Lustre: DEBUG MARKER: == sanity test 32d: open d32d/ext2-mountpoint/../d2/test_dir ========================================================== 00:03:56 (1734393836)
[ 1973.898840] loop0: detected capacity change from 0 to 16000
[ 1973.900255] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1973.948167] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1975.818214] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32e: stat d32e\/symlink-\>tmp\/symlink-\>lustre-subdir ========================================================== 00:03:59 \(1734393839\)
[ 1976.178745] Lustre: DEBUG MARKER: == sanity test 32e: stat d32e/symlink->tmp/symlink->lustre-subdir ========================================================== 00:03:59 (1734393839)
[ 1977.973347] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32f: open d32f\/symlink-\>tmp\/symlink-\>lustre-subdir ========================================================== 00:04:01 \(1734393841\)
[ 1978.334095] Lustre: DEBUG MARKER: == sanity test 32f: open d32f/symlink->tmp/symlink->lustre-subdir ========================================================== 00:04:01 (1734393841)
[ 1980.145180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32g: stat d32g\/symlink-\>tmp\/symlink-\>lustre-subdir\/2 ========================================================== 00:04:03 \(1734393843\)
[ 1980.515899] Lustre: DEBUG MARKER: == sanity test 32g: stat d32g/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 00:04:03 (1734393843)
[ 1980.703578] Autotest: Test running for 30 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 1982.337618] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32h: open d32h\/symlink-\>tmp\/symlink-\>lustre-subdir\/2 ========================================================== 00:04:05 \(1734393845\)
[ 1982.697314] Lustre: DEBUG MARKER: == sanity test 32h: open d32h/symlink->tmp/symlink->lustre-subdir/2 ========================================================== 00:04:05 (1734393845)
[ 1984.510975] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32i: stat d32i\/ext2-mountpoint\/..\/test_file ===================================================================== 00:04:07 \(1734393847\)
[ 1984.870799] Lustre: DEBUG MARKER: == sanity test 32i: stat d32i/ext2-mountpoint/../test_file ===================================================================== 00:04:07 (1734393847)
[ 1984.901025] loop0: detected capacity change from 0 to 16000
[ 1984.902467] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1985.009813] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1986.870459] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32j: open d32j\/ext2-mountpoint\/..\/test_file ===================================================================== 00:04:10 \(1734393850\)
[ 1987.228531] Lustre: lustre-OST0003-osc-ffff988d17847800: disconnect after 22s idle
[ 1987.229730] Lustre: DEBUG MARKER: == sanity test 32j: open d32j/ext2-mountpoint/../test_file ===================================================================== 00:04:10 (1734393850)
[ 1987.230045] Lustre: Skipped 10 previous similar messages
[ 1987.262656] loop0: detected capacity change from 0 to 16000
[ 1987.264041] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1987.293206] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1989.131264] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32k: stat d32k\/ext2-mountpoint\/..\/d2\/test_file ================================================================== 00:04:12 \(1734393852\)
[ 1989.495110] Lustre: DEBUG MARKER: == sanity test 32k: stat d32k/ext2-mountpoint/../d2/test_file ================================================================== 00:04:12 (1734393852)
[ 1989.531390] loop0: detected capacity change from 0 to 16000
[ 1989.533009] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1989.702613] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1991.569717] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32l: open d32l\/ext2-mountpoint\/..\/d2\/test_file ================================================================== 00:04:15 \(1734393855\)
[ 1991.931079] Lustre: DEBUG MARKER: == sanity test 32l: open d32l/ext2-mountpoint/../d2/test_file ================================================================== 00:04:15 (1734393855)
[ 1991.963258] loop0: detected capacity change from 0 to 16000
[ 1991.964773] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 1992.087485] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 1993.999651] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32m: stat d32m\/symlink-\>tmp\/symlink-\>lustre-root ================================================================ 00:04:17 \(1734393857\)
[ 1994.360440] Lustre: DEBUG MARKER: == sanity test 32m: stat d32m/symlink->tmp/symlink->lustre-root ================================================================ 00:04:17 (1734393857)
[ 1996.156064] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32n: open d32n\/symlink-\>tmp\/symlink-\>lustre-root ================================================================ 00:04:19 \(1734393859\)
[ 1996.514225] Lustre: DEBUG MARKER: == sanity test 32n: open d32n/symlink->tmp/symlink->lustre-root ================================================================ 00:04:19 (1734393859)
[ 1998.317382] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32o: stat d32o\/symlink-\>tmp\/symlink-\>lustre-root\/ ========================================================== 00:04:21 \(1734393861\)
[ 1998.679234] Lustre: DEBUG MARKER: == sanity test 32o: stat d32o/symlink->tmp/symlink->lustre-root/ ========================================================== 00:04:21 (1734393861)
[ 2000.475973] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32p: open d32p\/symlink-\>tmp\/symlink-\>lustre-root\/ ========================================================== 00:04:23 \(1734393863\)
[ 2000.834795] Lustre: DEBUG MARKER: == sanity test 32p: open d32p/symlink->tmp/symlink->lustre-root/ ========================================================== 00:04:23 (1734393863)
[ 2001.236322] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_1
[ 2001.589470] Lustre: DEBUG MARKER: 32p_1
[ 2001.991862] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_2
[ 2002.347030] Lustre: DEBUG MARKER: 32p_2
[ 2002.752172] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_3
[ 2003.108211] Lustre: DEBUG MARKER: 32p_3
[ 2003.673021] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_4
[ 2004.046549] Lustre: DEBUG MARKER: 32p_4
[ 2004.470929] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_5
[ 2004.871766] Lustre: DEBUG MARKER: 32p_5
[ 2005.279008] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_6
[ 2005.637416] Lustre: DEBUG MARKER: 32p_6
[ 2006.042005] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_7
[ 2006.401697] Lustre: DEBUG MARKER: 32p_7
[ 2006.806786] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_8
[ 2007.171192] Lustre: DEBUG MARKER: 32p_8
[ 2007.594363] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_9
[ 2007.961267] Lustre: DEBUG MARKER: 32p_9
[ 2008.377782] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 32p_10
[ 2008.736118] Lustre: DEBUG MARKER: 32p_10
[ 2015.527463] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32q: stat follows mountpoints in Lustre \(should return error\) ========================================================== 00:04:38 \(1734393878\)
[ 2015.895093] Lustre: DEBUG MARKER: == sanity test 32q: stat follows mountpoints in Lustre (should return error) ========================================================== 00:04:38 (1734393878)
[ 2015.929649] loop0: detected capacity change from 0 to 16000
[ 2015.931084] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2015.948779] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2017.904641] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 32r: opendir follows mountpoints in Lustre \(should return error\) ========================================================== 00:04:41 \(1734393881\)
[ 2018.265573] Lustre: DEBUG MARKER: == sanity test 32r: opendir follows mountpoints in Lustre (should return error) ========================================================== 00:04:41 (1734393881)
[ 2018.371643] loop0: detected capacity change from 0 to 16000
[ 2018.373086] EXT4-fs (loop0): mounting ext2 file system using the ext4 subsystem
[ 2018.400044] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 2020.243234] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33aa: write file with mode 444 \(should return error\) ========================================================== 00:04:43 \(1734393883\)
[ 2020.600697] Lustre: DEBUG MARKER: == sanity test 33aa: write file with mode 444 (should return error) ========================================================== 00:04:43 (1734393883)
[ 2021.025979] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 33_1
[ 2021.393538] Lustre: DEBUG MARKER: 33_1
[ 2021.802164] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 33_2
[ 2022.158731] Lustre: DEBUG MARKER: 33_2
[ 2023.918642] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33a: test open file\(mode=0444\) with O_RDWR \(should return error\) ========================================================== 00:04:47 \(1734393887\)
[ 2024.277924] Lustre: DEBUG MARKER: == sanity test 33a: test open file(mode=0444) with O_RDWR (should return error) ========================================================== 00:04:47 (1734393887)
[ 2026.064327] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33b: test open file with malformed flags \(No panic\) ========================================================== 00:04:49 \(1734393889\)
[ 2026.424923] Lustre: DEBUG MARKER: == sanity test 33b: test open file with malformed flags (No panic) ========================================================== 00:04:49 (1734393889)
[ 2028.216331] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33c: test write_bytes stats =============== 00:04:51 \(1734393891\)
[ 2028.578926] Lustre: DEBUG MARKER: == sanity test 33c: test write_bytes stats =============== 00:04:51 (1734393891)
[ 2031.505290] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33d: openfile with 444 modes and malformed flags under remote dir ========================================================== 00:04:54 \(1734393894\)
[ 2031.868320] Lustre: DEBUG MARKER: == sanity test 33d: openfile with 444 modes and malformed flags under remote dir ========================================================== 00:04:54 (1734393894)
[ 2032.271908] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_33d needs \>= 2 MDTs
[ 2032.644301] Lustre: DEBUG MARKER: SKIP: sanity test_33d needs >= 2 MDTs
[ 2033.074733] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33e: mkdir and striped directory should have same mode ========================================================== 00:04:56 \(1734393896\)
[ 2033.437342] Lustre: DEBUG MARKER: == sanity test 33e: mkdir and striped directory should have same mode ========================================================== 00:04:56 (1734393896)
[ 2033.842756] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_33e needs \>= 2 MDTs
[ 2034.203568] Lustre: DEBUG MARKER: SKIP: sanity test_33e needs >= 2 MDTs
[ 2034.636990] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33f: nonroot user can create, access, and remove a striped directory ========================================================== 00:04:58 \(1734393898\)
[ 2035.002871] Lustre: DEBUG MARKER: == sanity test 33f: nonroot user can create, access, and remove a striped directory ========================================================== 00:04:58 (1734393898)
[ 2035.406062] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_33f needs \>= 2 MDTs
[ 2035.766490] Lustre: DEBUG MARKER: SKIP: sanity test_33f needs >= 2 MDTs
[ 2036.204948] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33g: nonroot user create already existing root created file ========================================================== 00:04:59 \(1734393899\)
[ 2036.564644] Lustre: DEBUG MARKER: == sanity test 33g: nonroot user create already existing root created file ========================================================== 00:04:59 (1734393899)
[ 2038.338518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33h: temp file is located on the same MDT as target \(crush\) ========================================================== 00:05:01 \(1734393901\)
[ 2038.700862] Lustre: DEBUG MARKER: == sanity test 33h: temp file is located on the same MDT as target (crush) ========================================================== 00:05:01 (1734393901)
[ 2039.102999] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_33h needs \>= 2 MDTs
[ 2039.462223] Lustre: DEBUG MARKER: SKIP: sanity test_33h needs >= 2 MDTs
[ 2039.904466] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33hh: temp file is located on the same MDT as target \(crush2\) ========================================================== 00:05:03 \(1734393903\)
[ 2040.266611] Lustre: DEBUG MARKER: == sanity test 33hh: temp file is located on the same MDT as target (crush2) ========================================================== 00:05:03 (1734393903)
[ 2040.669672] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_33hh needs \>= 2 MDTs
[ 2041.025275] Lustre: DEBUG MARKER: SKIP: sanity test_33hh needs >= 2 MDTs
[ 2041.457556] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33i: striped directory can be accessed when one MDT is down ========================================================== 00:05:04 \(1734393904\)
[ 2041.813995] Lustre: DEBUG MARKER: == sanity test 33i: striped directory can be accessed when one MDT is down ========================================================== 00:05:04 (1734393904)
[ 2042.218985] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_33i needs \>= 2 MDTs
[ 2042.580916] Lustre: DEBUG MARKER: SKIP: sanity test_33i needs >= 2 MDTs
[ 2043.009114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 33j: lfs setdirstripe -D -i x,y,x should fail ========================================================== 00:05:06 \(1734393906\)
[ 2043.368871] Lustre: DEBUG MARKER: == sanity test 33j: lfs setdirstripe -D -i x,y,x should fail ========================================================== 00:05:06 (1734393906)
[ 2043.769537] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_33j needs \>= 2 MDTs
[ 2044.127998] Lustre: DEBUG MARKER: SKIP: sanity test_33j needs >= 2 MDTs
[ 2044.556307] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34a: truncate file that has not been opened ===================================================================== 00:05:08 \(1734393908\)
[ 2044.914417] Lustre: DEBUG MARKER: == sanity test 34a: truncate file that has not been opened ===================================================================== 00:05:08 (1734393908)
[ 2046.692824] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34b: O_RDONLY opening file doesn\'t create objects =============================================================== 00:05:10 \(1734393910\)
[ 2047.049886] Lustre: DEBUG MARKER: == sanity test 34b: O_RDONLY opening file doesn't create objects =============================================================== 00:05:10 (1734393910)
[ 2048.809644] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34c: O_RDWR opening file-with-size works ======================================================================== 00:05:12 \(1734393912\)
[ 2049.167232] Lustre: DEBUG MARKER: == sanity test 34c: O_RDWR opening file-with-size works ======================================================================== 00:05:12 (1734393912)
[ 2055.963042] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34d: write to sparse file ======================================================================================= 00:05:14 \(1734393914\)
[ 2056.322041] Lustre: DEBUG MARKER: == sanity test 34d: write to sparse file ======================================================================================= 00:05:14 (1734393914)
[ 2058.097873] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34e: create objects, some with size and some without ============================================================ 00:05:21 \(1734393921\)
[ 2058.460081] Lustre: DEBUG MARKER: == sanity test 34e: create objects, some with size and some without ============================================================ 00:05:21 (1734393921)
[ 2060.269815] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34f: read from a file with no objects until EOF ================================================================= 00:05:23 \(1734393923\)
[ 2060.636380] Lustre: DEBUG MARKER: == sanity test 34f: read from a file with no objects until EOF ================================================================= 00:05:23 (1734393923)
[ 2062.453497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34g: truncate long file ========================================================================================= 00:05:25 \(1734393925\)
[ 2062.824255] Lustre: DEBUG MARKER: == sanity test 34g: truncate long file ========================================================================================= 00:05:25 (1734393925)
[ 2064.642239] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 34h: ftruncate file under grouplock should not block ========================================================== 00:05:28 \(1734393928\)
[ 2065.002494] Lustre: DEBUG MARKER: == sanity test 34h: ftruncate file under grouplock should not block ========================================================== 00:05:28 (1734393928)
[ 2069.631039] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 35a: exec file with mode 444 \(should return and not leak\) ========================================================== 00:05:33 \(1734393933\)
[ 2069.986374] Lustre: DEBUG MARKER: == sanity test 35a: exec file with mode 444 (should return and not leak) ========================================================== 00:05:33 (1734393933)
[ 2071.796103] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36a: MDS utime check \(mknod, utime\) ======= 00:05:35 \(1734393935\)
[ 2072.156863] Lustre: DEBUG MARKER: == sanity test 36a: MDS utime check (mknod, utime) ======= 00:05:35 (1734393935)
[ 2073.922069] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36b: OST utime check \(open, utime\) ======== 00:05:37 \(1734393937\)
[ 2074.277703] Lustre: DEBUG MARKER: == sanity test 36b: OST utime check (open, utime) ======== 00:05:37 (1734393937)
[ 2076.051998] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36c: non-root MDS utime check \(mknod, utime\) ========================================================== 00:05:39 \(1734393939\)
[ 2076.415703] Lustre: DEBUG MARKER: == sanity test 36c: non-root MDS utime check (mknod, utime) ========================================================== 00:05:39 (1734393939)
[ 2078.190630] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36d: non-root OST utime check \(open, utime\) ========================================================== 00:05:41 \(1734393941\)
[ 2078.548814] Lustre: DEBUG MARKER: == sanity test 36d: non-root OST utime check (open, utime) ========================================================== 00:05:41 (1734393941)
[ 2080.327097] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36e: utime on non-owned file \(should return error\) ========================================================== 00:05:43 \(1734393943\)
[ 2080.686030] Lustre: DEBUG MARKER: == sanity test 36e: utime on non-owned file (should return error) ========================================================== 00:05:43 (1734393943)
[ 2082.476832] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 00:05:45 \(1734393945\)
[ 2082.853069] Lustre: DEBUG MARKER: == sanity test 36f: utime on file racing with OST BRW write ==================================================================== 00:05:45 (1734393945)
[ 2085.683528] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36g: FMD cache expiry =============================================================================== 00:05:49 \(1734393949\)
[ 2086.041508] Lustre: DEBUG MARKER: == sanity test 36g: FMD cache expiry =============================================================================== 00:05:49 (1734393949)
[ 2130.973640] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 00:06:34 \(1734393994\)
[ 2131.330227] Lustre: DEBUG MARKER: == sanity test 36h: utime on file racing with OST BRW write ==================================================================== 00:06:34 (1734393994)
[ 2134.150629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 36i: change mtime on striped directory ==== 00:06:37 \(1734393997\)
[ 2134.513010] Lustre: DEBUG MARKER: == sanity test 36i: change mtime on striped directory ==== 00:06:37 (1734393997)
[ 2134.915975] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_36i needs \>= 2 MDTs
[ 2135.273006] Lustre: DEBUG MARKER: SKIP: sanity test_36i needs >= 2 MDTs
[ 2135.704285] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 38: open a regular file with O_DIRECTORY should return -ENOTDIR ============================================================= 00:06:39 \(1734393999\)
[ 2136.065087] Lustre: DEBUG MARKER: == sanity test 38: open a regular file with O_DIRECTORY should return -ENOTDIR ============================================================= 00:06:39 (1734393999)
[ 2138.134814] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39a: mtime changed on create ============== 00:06:41 \(1734394001\)
[ 2138.497323] Lustre: DEBUG MARKER: == sanity test 39a: mtime changed on create ============== 00:06:41 (1734394001)
[ 2142.424906] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39b: mtime change on open, link, unlink, rename  ================================================================ 00:06:45 \(1734394005\)
[ 2142.792205] Lustre: DEBUG MARKER: == sanity test 39b: mtime change on open, link, unlink, rename ================================================================ 00:06:45 (1734394005)
[ 2145.988192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39c: mtime change on rename ===================================================================================== 00:06:49 \(1734394009\)
[ 2146.345713] Lustre: DEBUG MARKER: == sanity test 39c: mtime change on rename ===================================================================================== 00:06:49 (1734394009)
[ 2150.175515] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39d: create, utime, stat ======================================================================================== 00:06:53 \(1734394013\)
[ 2150.535092] Lustre: DEBUG MARKER: == sanity test 39d: create, utime, stat ======================================================================================== 00:06:53 (1734394013)
[ 2152.320926] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39e: create, stat, utime, stat ================================================================================== 00:06:55 \(1734394015\)
[ 2152.685119] Lustre: DEBUG MARKER: == sanity test 39e: create, stat, utime, stat ================================================================================== 00:06:55 (1734394015)
[ 2154.495675] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39f: create, stat, sleep, utime, stat =========================================================================== 00:06:57 \(1734394017\)
[ 2154.857609] Lustre: DEBUG MARKER: == sanity test 39f: create, stat, sleep, utime, stat =========================================================================== 00:06:57 (1734394017)
[ 2158.657892] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39g: write, chmod, stat ========================================================================================= 00:07:02 \(1734394022\)
[ 2159.020349] Lustre: DEBUG MARKER: == sanity test 39g: write, chmod, stat ========================================================================================= 00:07:02 (1734394022)
[ 2163.178045] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39h: write, utime within one second, stat ======================================================================= 00:07:06 \(1734394026\)
[ 2163.537499] Lustre: DEBUG MARKER: == sanity test 39h: write, utime within one second, stat ======================================================================= 00:07:06 (1734394026)
[ 2166.350595] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39i: write, rename, stat ======================================================================================== 00:07:09 \(1734394029\)
[ 2166.710736] Lustre: DEBUG MARKER: == sanity test 39i: write, rename, stat ======================================================================================== 00:07:09 (1734394029)
[ 2169.560772] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39j: write, rename, close, stat ================================================================================= 00:07:13 \(1734394033\)
[ 2169.918960] Lustre: DEBUG MARKER: == sanity test 39j: write, rename, close, stat ================================================================================= 00:07:13 (1734394033)
[ 2170.623681] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 debug_mb=150
[ 2172.468770] LustreError: 150145:0:(osc_io.c:901:osc_io_write_start()) cfs_fail_timeout id 412 sleeping for 1000ms
[ 2173.511248] LustreError: 150145:0:(osc_io.c:901:osc_io_write_start()) cfs_fail_timeout id 412 awake
[ 2173.982519] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug_mb=11
[ 2174.665041] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 2177.167666] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39k: write, utime, close, stat ================================================================================== 00:07:20 \(1734394040\)
[ 2177.524601] Lustre: DEBUG MARKER: == sanity test 39k: write, utime, close, stat ================================================================================== 00:07:20 (1734394040)
[ 2180.982758] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39l: directory atime update ===================================================================================== 00:07:24 \(1734394044\)
[ 2181.342466] Lustre: DEBUG MARKER: == sanity test 39l: directory atime update ===================================================================================== 00:07:24 (1734394044)
[ 2195.527846] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39m: test atime and mtime before 1970 ===== 00:07:38 \(1734394058\)
[ 2195.888414] Lustre: DEBUG MARKER: == sanity test 39m: test atime and mtime before 1970 ===== 00:07:38 (1734394058)
[ 2199.820266] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39n: check that O_NOATIME is honored ====== 00:07:43 \(1734394063\)
[ 2200.184681] Lustre: DEBUG MARKER: == sanity test 39n: check that O_NOATIME is honored ====== 00:07:43 (1734394063)
[ 2213.098799] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39o: directory cached attributes updated after create ========================================================== 00:07:56 \(1734394076\)
[ 2213.462471] Lustre: DEBUG MARKER: == sanity test 39o: directory cached attributes updated after create ========================================================== 00:07:56 (1734394076)
[ 2215.325514] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39p: remote directory cached attributes updated after create ================================================================== 00:07:58 \(1734394078\)
[ 2215.690616] Lustre: DEBUG MARKER: == sanity test 39p: remote directory cached attributes updated after create ================================================================== 00:07:58 (1734394078)
[ 2216.099852] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_39p needs \>= 2 MDTs
[ 2221.462380] Lustre: DEBUG MARKER: SKIP: sanity test_39p needs >= 2 MDTs
[ 2221.900585] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39r: lazy atime update on OST ============= 00:08:05 \(1734394085\)
[ 2222.259197] Lustre: DEBUG MARKER: == sanity test 39r: lazy atime update on OST ============= 00:08:05 (1734394085)
[ 2236.505891] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39q: close won\'t zero out atime =========== 00:08:19 \(1734394099\)
[ 2236.870577] Lustre: DEBUG MARKER: == sanity test 39q: close won't zero out atime =========== 00:08:19 (1734394099)
[ 2238.644083] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39s: relatime is supported ================ 00:08:22 \(1734394102\)
[ 2239.004991] Lustre: DEBUG MARKER: == sanity test 39s: relatime is supported ================ 00:08:22 (1734394102)
[ 2239.018306] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2239.029805] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2239.198747] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 2239.306625] Lustre: Unmounted lustre-client
[ 2239.317276] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2239.328403] Lustre: DEBUG MARKER: mount -t lustre -o relatime 10.240.22.189@tcp:/lustre /mnt/lustre
[ 2239.379484] Lustre: Mounted lustre-client
[ 2244.567183] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 2244.578502] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 2244.743533] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 2244.857511] Lustre: Unmounted lustre-client
[ 2244.864969] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 2244.875198] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 2244.924708] Lustre: Mounted lustre-client
[ 2246.715952] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 39u: stat race ============================ 00:08:30 \(1734394110\)
[ 2247.086188] Lustre: DEBUG MARKER: == sanity test 39u: stat race ============================ 00:08:30 (1734394110)
[ 2249.170077] Lustre: *** cfs_fail_loc=1434, val=5173***
[ 2249.171218] LustreError: 155083:0:(file.c:6211:ll_getattr_dentry()) cfs_race id 1435 sleeping
[ 2250.169524] LustreError: 155085:0:(file.c:1514:ll_merge_attr_nolock()) cfs_race id 1435 sleeping
[ 2254.234786] LustreError: 155083:0:(file.c:6211:ll_getattr_dentry()) cfs_fail_race id 1435 awake: rc=0
[ 2255.259415] LustreError: 155085:0:(file.c:1514:ll_merge_attr_nolock()) cfs_fail_race id 1435 awake: rc=0
[ 2255.261369] LustreError: 155085:0:(file.c:1519:ll_merge_attr_nolock()) cfs_fail_timeout id 1435 sleeping for 1000ms
[ 2256.302749] LustreError: 155085:0:(file.c:1519:ll_merge_attr_nolock()) cfs_fail_timeout id 1435 awake
[ 2256.304622] LustreError: 155085:0:(file.c:6211:ll_getattr_dentry()) cfs_race id 1435 sleeping
[ 2261.402731] LustreError: 155085:0:(file.c:6211:ll_getattr_dentry()) cfs_fail_race id 1435 awake: rc=0
[ 2263.257170] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 40: failed open\(O_TRUNC\) doesn\'t truncate ======================================================================= 00:08:46 \(1734394126\)
[ 2263.621670] Lustre: DEBUG MARKER: == sanity test 40: failed open(O_TRUNC) doesn't truncate ======================================================================= 00:08:46 (1734394126)
[ 2265.403078] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 41: test small file write + fstat =============================================================================== 00:08:48 \(1734394128\)
[ 2265.768489] Lustre: DEBUG MARKER: == sanity test 41: test small file write + fstat =============================================================================== 00:08:48 (1734394128)
[ 2278.528902] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_42a skipping ALWAYS excluded test 42a
[ 2278.892406] Lustre: DEBUG MARKER: SKIP: sanity test_42a skipping ALWAYS excluded test 42a
[ 2279.309625] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_42b skipping ALWAYS excluded test 42b
[ 2279.671081] Lustre: DEBUG MARKER: SKIP: sanity test_42b skipping ALWAYS excluded test 42b
[ 2280.082118] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_42c skipping ALWAYS excluded test 42c
[ 2280.459077] Lustre: DEBUG MARKER: SKIP: sanity test_42c skipping ALWAYS excluded test 42c
[ 2280.887653] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 00:09:04 \(1734394144\)
[ 2281.249240] Lustre: DEBUG MARKER: == sanity test 42d: test complete truncate of file with cached dirty data ========================================================== 00:09:04 (1734394144)
[ 2282.663358] Lustre: DEBUG MARKER: sync
[ 2286.363814] Autotest: Test running for 35 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2289.153476] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 2289.863727] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 2296.923229] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 00:09:20 \(1734394160\)
[ 2297.285533] Lustre: DEBUG MARKER: == sanity test 42e: verify sub-RPC writes are not done synchronously ========================================================== 00:09:20 (1734394160)
[ 2297.967803] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 2305.573216] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 2306.976719] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 2309.383358] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 2373.045705] Lustre: DEBUG MARKER: sync
[ 2374.402738] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 2375.104665] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 2377.159968] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 00:10:40 \(1734394240\)
[ 2377.526136] Lustre: DEBUG MARKER: == sanity test 43A: execution of file opened for write should return -ETXTBSY ========================================================== 00:10:40 (1734394240)
[ 2380.338093] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43a: open\(RDWR\) of file being executed should return -ETXTBSY ========================================================== 00:10:43 \(1734394243\)
[ 2380.708903] Lustre: DEBUG MARKER: == sanity test 43a: open(RDWR) of file being executed should return -ETXTBSY ========================================================== 00:10:43 (1734394243)
[ 2388.516647] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 00:10:51 \(1734394251\)
[ 2388.878473] Lustre: DEBUG MARKER: == sanity test 43b: truncate of file being executed should return -ETXTBSY ========================================================== 00:10:51 (1734394251)
[ 2391.696029] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 43c: md5sum of copy into lustre =========== 00:10:55 \(1734394255\)
[ 2392.051796] Lustre: DEBUG MARKER: == sanity test 43c: md5sum of copy into lustre =========== 00:10:55 (1734394255)
[ 2398.908908] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44A: zero length read from a sparse stripe ========================================================== 00:11:02 \(1734394262\)
[ 2399.267791] Lustre: DEBUG MARKER: == sanity test 44A: zero length read from a sparse stripe ========================================================== 00:11:02 (1734394262)
[ 2401.082400] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44a: test sparse pwrite ========================================================================================= 00:11:04 \(1734394264\)
[ 2401.447743] Lustre: DEBUG MARKER: == sanity test 44a: test sparse pwrite ========================================================================================= 00:11:04 (1734394264)
[ 2404.088402] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44b: write one byte at offset 0xfffffffe000 ========================================================== 00:11:07 \(1734394267\)
[ 2404.449503] Lustre: DEBUG MARKER: == sanity test 44b: write one byte at offset 0xfffffffe000 ========================================================== 00:11:07 (1734394267)
[ 2406.272435] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44c: write 1 byte at max_object_bytes - 1 offset ========================================================== 00:11:09 \(1734394269\)
[ 2406.630200] Lustre: DEBUG MARKER: == sanity test 44c: write 1 byte at max_object_bytes - 1 offset ========================================================== 00:11:09 (1734394269)
[ 2408.454239] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44d: if write at position fails \(EFBIG\), so should do append ========================================================== 00:11:11 \(1734394271\)
[ 2408.820414] Lustre: DEBUG MARKER: == sanity test 44d: if write at position fails (EFBIG), so should do append ========================================================== 00:11:11 (1734394271)
[ 2415.653985] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44e: write and read maximal stripes ======= 00:11:19 \(1734394279\)
[ 2416.016981] Lustre: DEBUG MARKER: == sanity test 44e: write and read maximal stripes ======= 00:11:19 (1734394279)
[ 2422.366049] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 44f: Check fiemap for sparse files ======== 00:11:25 \(1734394285\)
[ 2422.741798] Lustre: DEBUG MARKER: == sanity test 44f: Check fiemap for sparse files ======== 00:11:25 (1734394285)
[ 2587.072054] Autotest: Test running for 40 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 2887.906350] Autotest: Test running for 45 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3188.756668] Autotest: Test running for 50 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3489.567026] Autotest: Test running for 55 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3531.408001] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 45: osc io page accounting ================ 00:29:54 \(1734395394\)
[ 3531.774551] Lustre: DEBUG MARKER: == sanity test 45: osc io page accounting ================ 00:29:54 (1734395394)
[ 3535.867223] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 46: dirtying a previously written page ========================================================================== 00:29:59 \(1734395399\)
[ 3536.227408] Lustre: DEBUG MARKER: == sanity test 46: dirtying a previously written page ========================================================================== 00:29:59 (1734395399)
[ 3538.429644] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48a: Access renamed working dir \(should return errors\)=========================================================== 00:30:01 \(1734395401\)
[ 3538.792088] Lustre: DEBUG MARKER: == sanity test 48a: Access renamed working dir (should return errors)=========================================================== 00:30:01 (1734395401)
[ 3540.721017] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48b: Access removed working dir \(should return errors\)=========================================================== 00:30:04 \(1734395404\)
[ 3541.082188] Lustre: DEBUG MARKER: == sanity test 48b: Access removed working dir (should return errors)=========================================================== 00:30:04 (1734395404)
[ 3542.911440] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48c: Access removed working subdir \(should return errors\) ========================================================== 00:30:06 \(1734395406\)
[ 3543.271181] Lustre: DEBUG MARKER: == sanity test 48c: Access removed working subdir (should return errors) ========================================================== 00:30:06 (1734395406)
[ 3545.121630] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48d: Access removed parent subdir \(should return errors\) ========================================================== 00:30:08 \(1734395408\)
[ 3545.483585] Lustre: DEBUG MARKER: == sanity test 48d: Access removed parent subdir (should return errors) ========================================================== 00:30:08 (1734395408)
[ 3547.332597] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48e: Access to recreated parent subdir \(should return errors\) ========================================================== 00:30:10 \(1734395410\)
[ 3547.691311] Lustre: DEBUG MARKER: == sanity test 48e: Access to recreated parent subdir (should return errors) ========================================================== 00:30:10 (1734395410)
[ 3549.541927] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 48f: non-zero nlink dir unlink won\'t LBUG\(\) ========================================================== 00:30:12 \(1734395412\)
[ 3549.948426] Lustre: DEBUG MARKER: == sanity test 48f: non-zero nlink dir unlink won't LBUG() ========================================================== 00:30:12 (1734395412)
[ 3550.369355] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_48f needs \>= 2 MDTs
[ 3550.732335] Lustre: DEBUG MARKER: SKIP: sanity test_48f needs >= 2 MDTs
[ 3551.198673] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 49: Change max_pages_per_rpc won\'t break osc extent ========================================================== 00:30:14 \(1734395414\)
[ 3551.561365] Lustre: DEBUG MARKER: == sanity test 49: Change max_pages_per_rpc won't break osc extent ========================================================== 00:30:14 (1734395414)
[ 3553.789868] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 50: special situations: \/proc symlinks  ========================================================================= 00:30:17 \(1734395417\)
[ 3554.172250] Lustre: DEBUG MARKER: == sanity test 50: special situations: /proc symlinks ========================================================================= 00:30:17 (1734395417)
[ 3555.988625] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51a: special situations: split htree with empty entry ============================================================ 00:30:19 \(1734395419\)
[ 3556.366047] Lustre: DEBUG MARKER: == sanity test 51a: special situations: split htree with empty entry ============================================================ 00:30:19 (1734395419)
[ 3557.095572] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3557.835568] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3560.554857] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51b: exceed 64k subdirectory nlink limit on create, verify unlink ========================================================== 00:30:23 \(1734395423\)
[ 3560.954534] Lustre: DEBUG MARKER: == sanity test 51b: exceed 64k subdirectory nlink limit on create, verify unlink ========================================================== 00:30:23 (1734395423)
[ 3561.695269] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3580.902663] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3582.926155] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3612.524169] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3615.481942] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51d: check LOV round-robin OST object distribution ========================================================== 00:31:18 \(1734395478\)
[ 3615.843435] Lustre: DEBUG MARKER: == sanity test 51d: check LOV round-robin OST object distribution ========================================================== 00:31:18 (1734395478)
[ 3625.784586] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=3
[ 3626.150222] Lustre: DEBUG MARKER: create files with stripecount=3
[ 3626.863940] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3627.944775] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3629.795115] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 302 objects, 102 are index 0
[ 3630.170531] Lustre: DEBUG MARKER: OST0 has 302 objects, 102 are index 0
[ 3630.591594] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 302 objects, 100 are index 0
[ 3630.950153] Lustre: DEBUG MARKER: OST1 has 302 objects, 100 are index 0
[ 3631.367785] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 302 objects, 99 are index 0
[ 3631.729162] Lustre: DEBUG MARKER: OST2 has 302 objects, 99 are index 0
[ 3632.147572] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 302 objects, 101 are index 0
[ 3632.511037] Lustre: DEBUG MARKER: OST3 has 302 objects, 101 are index 0
[ 3632.933918] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 302 objects, 100 are index 0
[ 3633.304861] Lustre: DEBUG MARKER: OST4 has 302 objects, 100 are index 0
[ 3633.720840] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 303 objects, 100 are index 0
[ 3634.078664] Lustre: DEBUG MARKER: OST5 has 303 objects, 100 are index 0
[ 3634.496899] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 301 objects, 100 are index 0
[ 3634.853736] Lustre: DEBUG MARKER: OST6 has 301 objects, 100 are index 0
[ 3635.516037] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3636.408579] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3645.568620] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=4
[ 3645.953795] Lustre: DEBUG MARKER: create files with stripecount=4
[ 3646.638793] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3647.672549] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3650.170235] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 403 objects, 101 are index 0
[ 3650.531594] Lustre: DEBUG MARKER: OST0 has 403 objects, 101 are index 0
[ 3650.948241] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 401 objects, 100 are index 0
[ 3651.308653] Lustre: DEBUG MARKER: OST1 has 401 objects, 100 are index 0
[ 3651.730579] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 403 objects, 100 are index 0
[ 3652.090671] Lustre: DEBUG MARKER: OST2 has 403 objects, 100 are index 0
[ 3652.514215] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 401 objects, 99 are index 0
[ 3652.873672] Lustre: DEBUG MARKER: OST3 has 401 objects, 99 are index 0
[ 3653.304377] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 403 objects, 102 are index 0
[ 3653.665122] Lustre: DEBUG MARKER: OST4 has 403 objects, 102 are index 0
[ 3654.095043] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 401 objects, 98 are index 0
[ 3654.456997] Lustre: DEBUG MARKER: OST5 has 401 objects, 98 are index 0
[ 3654.878531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 402 objects, 102 are index 0
[ 3655.239517] Lustre: DEBUG MARKER: OST6 has 402 objects, 102 are index 0
[ 3655.910255] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3656.840697] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3666.019138] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=5
[ 3666.379471] Lustre: DEBUG MARKER: create files with stripecount=5
[ 3667.120199] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3668.184232] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3670.012628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 502 objects, 101 are index 0
[ 3670.382889] Lustre: DEBUG MARKER: OST0 has 502 objects, 101 are index 0
[ 3670.808945] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 503 objects, 101 are index 0
[ 3671.183234] Lustre: DEBUG MARKER: OST1 has 503 objects, 101 are index 0
[ 3671.614180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 503 objects, 99 are index 0
[ 3671.978358] Lustre: DEBUG MARKER: OST2 has 503 objects, 99 are index 0
[ 3672.404293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 502 objects, 100 are index 0
[ 3672.764470] Lustre: DEBUG MARKER: OST3 has 502 objects, 100 are index 0
[ 3673.188006] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 501 objects, 99 are index 0
[ 3673.570016] Lustre: DEBUG MARKER: OST4 has 501 objects, 99 are index 0
[ 3673.992638] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 502 objects, 101 are index 0
[ 3674.358113] Lustre: DEBUG MARKER: OST5 has 502 objects, 101 are index 0
[ 3674.780963] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 501 objects, 101 are index 0
[ 3675.144544] Lustre: DEBUG MARKER: OST6 has 501 objects, 101 are index 0
[ 3675.822165] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3676.764913] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3685.947929] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=6
[ 3686.312239] Lustre: DEBUG MARKER: create files with stripecount=6
[ 3686.996910] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3688.086157] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3689.951151] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 602 objects, 101 are index 0
[ 3690.311753] Lustre: DEBUG MARKER: OST0 has 602 objects, 101 are index 0
[ 3690.734911] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 602 objects, 100 are index 0
[ 3691.098802] Lustre: DEBUG MARKER: OST1 has 602 objects, 100 are index 0
[ 3691.520287] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 602 objects, 100 are index 0
[ 3696.887262] Lustre: DEBUG MARKER: OST2 has 602 objects, 100 are index 0
[ 3697.324318] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 602 objects, 100 are index 0
[ 3697.684269] Lustre: DEBUG MARKER: OST3 has 602 objects, 100 are index 0
[ 3698.106005] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 602 objects, 100 are index 0
[ 3698.463053] Lustre: DEBUG MARKER: OST4 has 602 objects, 100 are index 0
[ 3698.895600] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 602 objects, 100 are index 0
[ 3699.257803] Lustre: DEBUG MARKER: OST5 has 602 objects, 100 are index 0
[ 3699.693107] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 602 objects, 101 are index 0
[ 3700.049508] Lustre: DEBUG MARKER: OST6 has 602 objects, 101 are index 0
[ 3700.718176] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3701.642838] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3711.635717] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create files with stripecount=7
[ 3711.992302] Lustre: DEBUG MARKER: create files with stripecount=7
[ 3712.689405] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3713.772231] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3715.565906] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST0 has 702 objects, 101 are index 0
[ 3715.930064] Lustre: DEBUG MARKER: OST0 has 702 objects, 101 are index 0
[ 3716.355260] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST1 has 702 objects, 100 are index 0
[ 3716.717231] Lustre: DEBUG MARKER: OST1 has 702 objects, 100 are index 0
[ 3717.144287] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST2 has 702 objects, 100 are index 0
[ 3717.508063] Lustre: DEBUG MARKER: OST2 has 702 objects, 100 are index 0
[ 3717.936868] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST3 has 702 objects, 100 are index 0
[ 3718.295118] Lustre: DEBUG MARKER: OST3 has 702 objects, 100 are index 0
[ 3718.714380] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST4 has 702 objects, 100 are index 0
[ 3719.068469] Lustre: DEBUG MARKER: OST4 has 702 objects, 100 are index 0
[ 3719.487425] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST5 has 702 objects, 100 are index 0
[ 3719.844249] Lustre: DEBUG MARKER: OST5 has 702 objects, 100 are index 0
[ 3720.255345] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OST6 has 702 objects, 101 are index 0
[ 3720.612108] Lustre: DEBUG MARKER: OST6 has 702 objects, 101 are index 0
[ 3721.276469] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3722.179840] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3725.568275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51e: check file nlink limit =============== 00:33:09 \(1734395589\)
[ 3725.926510] Lustre: DEBUG MARKER: == sanity test 51e: check file nlink limit =============== 00:33:09 (1734395589)
[ 3730.973382] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3761.076258] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3762.648937] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3788.884965] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3791.183753] Autotest: Test running for 60 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 3791.613909] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 51f: check many open files limit ========== 00:34:15 \(1734395655\)
[ 3791.970161] Lustre: DEBUG MARKER: == sanity test 51f: check many open files limit ========== 00:34:15 (1734395655)
[ 3792.675564] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3843.849184] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3845.406687] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 3872.928951] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 3875.653866] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 52a: append-only flag test \(should return errors\) ========================================================== 00:35:39 \(1734395739\)
[ 3876.010395] Lustre: DEBUG MARKER: == sanity test 52a: append-only flag test (should return errors) ========================================================== 00:35:39 (1734395739)
[ 3876.639259] LustreError: 184113:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff988c12221000: inode [0x2000013a3:0xb0ba:0x0] mdc close failed: rc = -1
[ 3878.848096] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 52b: immutable flag test \(should return errors\) ================================================================= 00:35:42 \(1734395742\)
[ 3879.206239] Lustre: DEBUG MARKER: == sanity test 52b: immutable flag test (should return errors) ================================================================= 00:35:42 (1734395742)
[ 3886.123242] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 53: verify that MDS and OSTs agree on pre-creation ============================================================== 00:35:49 \(1734395749\)
[ 3886.487760] Lustre: DEBUG MARKER: == sanity test 53: verify that MDS and OSTs agree on pre-creation ============================================================== 00:35:49 (1734395749)
[ 3895.553186] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54a: unix domain socket test ============== 00:35:59 \(1734395759\)
[ 3895.918361] Lustre: DEBUG MARKER: == sanity test 54a: unix domain socket test ============== 00:35:59 (1734395759)
[ 3897.794811] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54b: char device works in lustre ================================================================================ 00:36:01 \(1734395761\)
[ 3898.160672] Lustre: DEBUG MARKER: == sanity test 54b: char device works in lustre ================================================================================ 00:36:01 (1734395761)
[ 3899.970888] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54c: block device works in lustre =============================================================================== 00:36:03 \(1734395763\)
[ 3900.339911] Lustre: DEBUG MARKER: == sanity test 54c: block device works in lustre =============================================================================== 00:36:03 (1734395763)
[ 3901.056728] loop3: detected capacity change from 0 to 8200
[ 3901.361663] blk_update_request: operation not supported error, dev loop3, sector 8064 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 3901.373168] blk_update_request: operation not supported error, dev loop3, sector 42 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 3901.431483] blk_update_request: operation not supported error, dev loop3, sector 580 op 0x9:(WRITE_ZEROES) flags 0x800800 phys_seg 0 prio class 0
[ 3902.817096] EXT4-fs (loop3): mounting ext2 file system using the ext4 subsystem
[ 3902.857714] EXT4-fs (loop3): mounted filesystem without journal. Opts: (null). Quota mode: none.
[ 3904.881774] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54d: fifo device works in lustre ================================================================================ 00:36:08 \(1734395768\)
[ 3905.243675] Lustre: DEBUG MARKER: == sanity test 54d: fifo device works in lustre ================================================================================ 00:36:08 (1734395768)
[ 3907.026473] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 54e: console\/tty device works in lustre ================================================================================ 00:36:10 \(1734395770\)
[ 3907.390367] Lustre: DEBUG MARKER: == sanity test 54e: console/tty device works in lustre ================================================================================ 00:36:10 (1734395770)
aaaaaa
[ 3909.198460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 55a: OBD device life cycle unit tests ===== 00:36:12 \(1734395772\)
[ 3909.560824] Lustre: DEBUG MARKER: == sanity test 55a: OBD device life cycle unit tests ===== 00:36:12 (1734395772)
[ 3909.591487] Lustre: OBD: obd_test_setup
[ 3909.591489] Lustre: OBD: obd_name: obd_name, obd_num: 11, obd_uuid: obd_uuid
[ 3909.592315] Lustre: OBD: class_name2dev(): 11, PASS
[ 3909.593654] Lustre: OBD: class_uuid2dev(): 11, PASS
[ 3909.594611] Lustre: OBD: class_name2obd(): 11, PASS
[ 3909.595569] Lustre: OBD: class_uuid2obd(): 11, PASS
[ 3909.604620] Lustre: OBD: obd_test_cleanup
[ 3911.413721] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 55b: Load and unload max OBD devices ====== 00:36:14 \(1734395774\)
[ 3911.771098] Lustre: DEBUG MARKER: == sanity test 55b: Load and unload max OBD devices ====== 00:36:14 (1734395774)
[ 4019.321641] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56a: check \/usr\/bin\/lfs getstripe ========= 00:38:02 \(1734395882\)
[ 4019.692349] Lustre: DEBUG MARKER: == sanity test 56a: check /usr/bin/lfs getstripe ========= 00:38:02 (1734395882)
[ 4021.956131] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56b: check \/usr\/bin\/lfs getdirstripe ====== 00:38:05 \(1734395885\)
[ 4022.363529] Lustre: DEBUG MARKER: == sanity test 56b: check /usr/bin/lfs getdirstripe ====== 00:38:05 (1734395885)
[ 4029.220266] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56bb: check \/usr\/bin\/lfs getdirstripe layout is YAML ========================================================== 00:38:12 \(1734395892\)
[ 4029.590769] Lustre: DEBUG MARKER: == sanity test 56bb: check /usr/bin/lfs getdirstripe layout is YAML ========================================================== 00:38:12 (1734395892)
[ 4031.692306] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56bc: check \'\/usr\/bin\/lfs getdirstripe --yaml\' params are valid ========================================================== 00:38:15 \(1734395895\)
[ 4032.060302] Lustre: DEBUG MARKER: == sanity test 56bc: check '/usr/bin/lfs getdirstripe --yaml' params are valid ========================================================== 00:38:15 (1734395895)
[ 4032.471271] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_56bc need \>= 2 MDTs
[ 4032.834647] Lustre: DEBUG MARKER: SKIP: sanity test_56bc need >= 2 MDTs
[ 4033.311607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56c: check \'lfs df\' showing device status ========================================================== 00:38:16 \(1734395896\)
[ 4033.678193] Lustre: DEBUG MARKER: == sanity test 56c: check 'lfs df' showing device status ========================================================== 00:38:16 (1734395896)
[ 4058.305163] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56d: \'lfs df -v\' prints only configured devices ========================================================== 00:38:41 \(1734395921\)
[ 4058.672245] Lustre: DEBUG MARKER: == sanity test 56d: 'lfs df -v' prints only configured devices ========================================================== 00:38:41 (1734395921)
[ 4065.463524] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56e: \'lfs df\' Handle non LustreFS & multiple LustreFS ========================================================== 00:38:43 \(1734395923\)
[ 4065.825922] Lustre: DEBUG MARKER: == sanity test 56e: 'lfs df' Handle non LustreFS
[ 4067.660353] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56g: check lfs find -name ================= 00:38:51 \(1734395931\)
[ 4068.027554] Lustre: DEBUG MARKER: == sanity test 56g: check lfs find -name ================= 00:38:51 (1734395931)
[ 4075.005168] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56h: check lfs find ! -name =============== 00:38:58 \(1734395938\)
[ 4075.373562] Lustre: DEBUG MARKER: == sanity test 56h: check lfs find ! -name =============== 00:38:58 (1734395938)
[ 4077.196810] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56i: check \'lfs find -ost UUID\' skips directories ========================================================== 00:39:00 \(1734395940\)
[ 4077.560540] Lustre: DEBUG MARKER: == sanity test 56i: check 'lfs find -ost UUID' skips directories ========================================================== 00:39:00 (1734395940)
[ 4079.383282] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ib: check \'lfs find -ost INDEX_RANGE\' command ========================================================== 00:39:02 \(1734395942\)
[ 4079.747618] Lustre: DEBUG MARKER: == sanity test 56ib: check 'lfs find -ost INDEX_RANGE' command ========================================================== 00:39:02 (1734395942)
[ 4086.573680] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56j: check lfs find -type d =============== 00:39:10 \(1734395950\)
[ 4086.949351] Lustre: DEBUG MARKER: == sanity test 56j: check lfs find -type d =============== 00:39:10 (1734395950)
[ 4092.220899] Autotest: Test running for 65 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 4093.811345] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56k: check lfs find -type f =============== 00:39:17 \(1734395957\)
[ 4094.172359] Lustre: DEBUG MARKER: == sanity test 56k: check lfs find -type f =============== 00:39:17 (1734395957)
[ 4101.008185] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56l: check lfs find -type b =============== 00:39:24 \(1734395964\)
[ 4101.373670] Lustre: DEBUG MARKER: == sanity test 56l: check lfs find -type b =============== 00:39:24 (1734395964)
[ 4103.207660] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56m: check lfs find -type c =============== 00:39:26 \(1734395966\)
[ 4103.572563] Lustre: DEBUG MARKER: == sanity test 56m: check lfs find -type c =============== 00:39:26 (1734395966)
[ 4105.400157] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56n: check lfs find -type l =============== 00:39:28 \(1734395968\)
[ 4105.766144] Lustre: DEBUG MARKER: == sanity test 56n: check lfs find -type l =============== 00:39:28 (1734395968)
[ 4112.571696] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56o: check lfs find -mtime for old files == 00:39:36 \(1734395976\)
[ 4112.933995] Lustre: DEBUG MARKER: == sanity test 56o: check lfs find -mtime for old files == 00:39:36 (1734395976)
[ 4116.672566] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ob: check lfs find -atime -mtime -ctime with units ========================================================== 00:39:40 \(1734395980\)
[ 4117.043279] Lustre: DEBUG MARKER: == sanity test 56ob: check lfs find -atime -mtime -ctime with units ========================================================== 00:39:40 (1734395980)
[ 4121.162067] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56oc: check lfs find -newerXY work ======== 00:39:44 \(1734395984\)
[ 4121.524702] Lustre: DEBUG MARKER: == sanity test 56oc: check lfs find -newerXY work ======== 00:39:44 (1734395984)
[ 4184.687690] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56od: check lfs find -btime with units ==== 00:40:48 \(1734396048\)
[ 4185.049236] Lustre: DEBUG MARKER: == sanity test 56od: check lfs find -btime with units ==== 00:40:48 (1734396048)
[ 4192.806765] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56oe: check lfs find with time range ====== 00:40:56 \(1734396056\)
[ 4193.170203] Lustre: DEBUG MARKER: == sanity test 56oe: check lfs find with time range ====== 00:40:56 (1734396056)
[ 4215.663968] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56p: check lfs find -uid and ! -uid ======= 00:41:19 \(1734396079\)
[ 4216.031144] Lustre: DEBUG MARKER: == sanity test 56p: check lfs find -uid and ! -uid ======= 00:41:19 (1734396079)
[ 4218.059135] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56q: check lfs find -gid and ! -gid ======= 00:41:21 \(1734396081\)
[ 4218.430573] Lustre: DEBUG MARKER: == sanity test 56q: check lfs find -gid and ! -gid ======= 00:41:21 (1734396081)
[ 4220.525117] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56r: check lfs find -size works =========== 00:41:23 \(1734396083\)
[ 4220.889275] Lustre: DEBUG MARKER: == sanity test 56r: check lfs find -size works =========== 00:41:23 (1734396083)
[ 4224.094447] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ra: check lfs find -size -lazy works for data on OSTs ========================================================== 00:41:27 \(1734396087\)
[ 4224.459535] Lustre: DEBUG MARKER: == sanity test 56ra: check lfs find -size -lazy works for data on OSTs ========================================================== 00:41:27 (1734396087)
[ 4227.712867] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56rb: check lfs find --size --ost\/--mdt works ========================================================== 00:41:31 \(1734396091\)
[ 4228.083271] Lustre: DEBUG MARKER: == sanity test 56rb: check lfs find --size --ost/--mdt works ========================================================== 00:41:31 (1734396091)
[ 4229.981712] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56rc: check lfs find --mdt-count\/--mdt-hash works ========================================================== 00:41:33 \(1734396093\)
[ 4230.347635] Lustre: DEBUG MARKER: == sanity test 56rc: check lfs find --mdt-count/--mdt-hash works ========================================================== 00:41:33 (1734396093)
[ 4230.774341] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_56rc needs at least 2 MDTs
[ 4231.138228] Lustre: DEBUG MARKER: SKIP: sanity test_56rc needs at least 2 MDTs
[ 4231.588257] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56rd: check lfs find --printf special files ========================================================== 00:41:35 \(1734396095\)
[ 4231.959043] Lustre: DEBUG MARKER: == sanity test 56rd: check lfs find --printf special files ========================================================== 00:41:35 (1734396095)
[ 4233.893283] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56s: check lfs find -stripe-count works === 00:41:37 \(1734396097\)
[ 4234.263023] Lustre: DEBUG MARKER: == sanity test 56s: check lfs find -stripe-count works === 00:41:37 (1734396097)
[ 4236.391607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56t: check lfs find -stripe-size works ==== 00:41:39 \(1734396099\)
[ 4236.764037] Lustre: DEBUG MARKER: == sanity test 56t: check lfs find -stripe-size works ==== 00:41:39 (1734396099)
[ 4239.069447] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56u: check lfs find -stripe-index works === 00:41:42 \(1734396102\)
[ 4239.453733] Lustre: DEBUG MARKER: == sanity test 56u: check lfs find -stripe-index works === 00:41:42 (1734396102)
[ 4241.605377] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56v: check \'lfs find -m match with lfs getstripe -m\' ========================================================== 00:41:45 \(1734396105\)
[ 4241.972752] Lustre: DEBUG MARKER: == sanity test 56v: check 'lfs find -m match with lfs getstripe -m' ========================================================== 00:41:45 (1734396105)
[ 4244.077693] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wa: check lfs_migrate -c stripe_count works ========================================================== 00:41:47 \(1734396107\)
[ 4244.446357] Lustre: DEBUG MARKER: == sanity test 56wa: check lfs_migrate -c stripe_count works ========================================================== 00:41:47 (1734396107)
[ 4258.645859] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4259.544354] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4289.788645] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wb: check lfs_migrate pool support ====== 00:42:33 \(1734396153\)
[ 4290.161147] Lustre: DEBUG MARKER: == sanity test 56wb: check lfs_migrate pool support ====== 00:42:33 (1734396153)
[ 4292.522672] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 		2>/dev/null || echo foo
[ 4294.876806] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool |
[ 4294.876806] 			grep -e lustre-OST0000_UUID  | sort -u | tr '\n' ' ' 
[ 4300.684003] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.testpool 		2>/dev/null || echo foo
[ 4302.615553] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wc: check unrecognized options for lfs_migrate are passed through ========================================================== 00:42:46 \(1734396166\)
[ 4302.979082] Lustre: DEBUG MARKER: == sanity test 56wc: check unrecognized options for lfs_migrate are passed through ========================================================== 00:42:46 (1734396166)
[ 4314.454724] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56wd: check lfs_migrate --rsync and --no-rsync work ========================================================== 00:42:57 \(1734396177\)
[ 4314.818395] Lustre: DEBUG MARKER: == sanity test 56wd: check lfs_migrate --rsync and --no-rsync work ========================================================== 00:42:57 (1734396177)
[ 4316.753809] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56we: check lfs_migrate --non-direct\|-D support ========================================================== 00:43:00 \(1734396180\)
[ 4317.115732] Lustre: DEBUG MARKER: == sanity test 56we: check lfs_migrate --non-direct|-D support ========================================================== 00:43:00 (1734396180)
[ 4319.186382] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56x: lfs migration support ================ 00:43:02 \(1734396182\)
[ 4319.547124] Lustre: DEBUG MARKER: == sanity test 56x: lfs migration support ================ 00:43:02 (1734396182)
[ 4321.596325] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xa: lfs migration --block support ======= 00:43:05 \(1734396185\)
[ 4321.957100] Lustre: DEBUG MARKER: == sanity test 56xa: lfs migration --block support ======= 00:43:05 (1734396185)
[ 4324.043487] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xb: lfs migration hard link support ===== 00:43:07 \(1734396187\)
[ 4324.401980] Lustre: DEBUG MARKER: == sanity test 56xb: lfs migration hard link support ===== 00:43:07 (1734396187)
[ 4353.328933] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xc: lfs migration autostripe ============ 00:43:36 \(1734396216\)
[ 4353.696716] Lustre: DEBUG MARKER: == sanity test 56xc: lfs migration autostripe ============ 00:43:36 (1734396216)
[ 4356.095202] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xd: check lfs_migrate --yaml and --copy support ========================================================== 00:43:39 \(1734396219\)
[ 4356.459214] Lustre: DEBUG MARKER: == sanity test 56xd: check lfs_migrate --yaml and --copy support ========================================================== 00:43:39 (1734396219)
[ 4360.387977] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xe: migrate a composite layout file ===== 00:43:43 \(1734396223\)
[ 4360.750100] Lustre: DEBUG MARKER: == sanity test 56xe: migrate a composite layout file ===== 00:43:43 (1734396223)
[ 4372.286613] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xf: FID is not lost during migration of a composite layout file ========================================================== 00:43:55 \(1734396235\)
[ 4372.652722] Lustre: DEBUG MARKER: == sanity test 56xf: FID is not lost during migration of a composite layout file ========================================================== 00:43:55 (1734396235)
[ 4376.059910] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xg: lfs migrate pool support ============ 00:43:59 \(1734396239\)
[ 4376.421653] Lustre: DEBUG MARKER: == sanity test 56xg: lfs migrate pool support ============ 00:43:59 (1734396239)
[ 4378.759281] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_0 		2>/dev/null || echo foo
[ 4380.105936] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_0 |
[ 4380.105936] 			grep -e lustre-OST0000_UUID  | sort -u | tr '\n' ' ' 
[ 4381.124880] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_0 |
[ 4381.124880] 			grep -e lustre-OST0000_UUID  | sort -u | tr '\n' ' ' 
[ 4382.458917] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_1 		2>/dev/null || echo foo
[ 4383.477003] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_1 		2>/dev/null || echo foo
[ 4385.827851] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_1 |
[ 4385.827851] 			grep -e lustre-OST0001_UUID  | sort -u | tr '\n' ' ' 
[ 4388.172713] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_2 		2>/dev/null || echo foo
[ 4390.530213] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_2 |
[ 4390.530213] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID  | sort -u | tr '\n' ' ' 
[ 4393.534419] Autotest: Test running for 70 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 4398.376025] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_0 		2>/dev/null || echo foo
[ 4403.943837] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_1 		2>/dev/null || echo foo
[ 4409.822482] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_2 		2>/dev/null || echo foo
[ 4410.839127] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56xg_2 		2>/dev/null || echo foo
[ 4412.750564] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xh: lfs migrate bandwidth limitation support ========================================================== 00:44:36 \(1734396276\)
[ 4413.110381] Lustre: DEBUG MARKER: == sanity test 56xh: lfs migrate bandwidth limitation support ========================================================== 00:44:36 (1734396276)
[ 4441.556757] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xi: lfs migrate stats support =========== 00:45:05 \(1734396305\)
[ 4441.922391] Lustre: DEBUG MARKER: == sanity test 56xi: lfs migrate stats support =========== 00:45:05 (1734396305)
[ 4450.545476] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xj: lfs migrate -b should not cause starvation of threads on OSS ========================================================== 00:45:13 \(1734396313\)
[ 4450.911509] Lustre: DEBUG MARKER: == sanity test 56xj: lfs migrate -b should not cause starvation of threads on OSS ========================================================== 00:45:13 (1734396313)
[ 4452.480914] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4453.237309] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4458.900531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xk: lfs mirror resync bandwidth limitation support ========================================================== 00:45:22 \(1734396322\)
[ 4459.269321] Lustre: DEBUG MARKER: == sanity test 56xk: lfs mirror resync bandwidth limitation support ========================================================== 00:45:22 (1734396322)
[ 4472.191799] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56xl: lfs mirror resync stats support ===== 00:45:30 \(1734396330\)
[ 4472.561808] Lustre: DEBUG MARKER: == sanity test 56xl: lfs mirror resync stats support ===== 00:45:30 (1734396330)
[ 4475.554294] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56y: lfs find -L raid0\|released =========== 00:45:39 \(1734396339\)
[ 4475.920140] Lustre: DEBUG MARKER: == sanity test 56y: lfs find -L raid0|released =========== 00:45:39 (1734396339)
[ 4477.794966] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56z: lfs find should continue after an error ========================================================== 00:45:41 \(1734396341\)
[ 4478.157626] Lustre: DEBUG MARKER: == sanity test 56z: lfs find should continue after an error ========================================================== 00:45:41 (1734396341)
[ 4480.247940] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56aa: lfs find --size under striped dir === 00:45:43 \(1734396343\)
[ 4480.611478] Lustre: DEBUG MARKER: == sanity test 56aa: lfs find --size under striped dir === 00:45:43 (1734396343)
[ 4481.020360] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_56aa needs \>= 2 MDTs
[ 4481.380870] Lustre: DEBUG MARKER: SKIP: sanity test_56aa needs >= 2 MDTs
[ 4481.822426] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ab: lfs find --blocks =================== 00:45:45 \(1734396345\)
[ 4482.184123] Lustre: DEBUG MARKER: == sanity test 56ab: lfs find --blocks =================== 00:45:45 (1734396345)
[ 4485.166175] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56aca: check lfs find -perm with octal representation ========================================================== 00:45:48 \(1734396348\)
[ 4485.536543] Lustre: DEBUG MARKER: == sanity test 56aca: check lfs find -perm with octal representation ========================================================== 00:45:48 (1734396348)
[ 4490.222628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56acb: check lfs find -perm with symbolic representation ========================================================== 00:45:53 \(1734396353\)
[ 4490.589611] Lustre: DEBUG MARKER: == sanity test 56acb: check lfs find -perm with symbolic representation ========================================================== 00:45:53 (1734396353)
[ 4498.118437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56acc: check parsing error for lfs find -perm ========================================================== 00:46:01 \(1734396361\)
[ 4498.486606] Lustre: DEBUG MARKER: == sanity test 56acc: check parsing error for lfs find -perm ========================================================== 00:46:01 (1734396361)
[ 4500.348030] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ba: test lfs find --component-end, -start, -count, and -flags ========================================================== 00:46:03 \(1734396363\)
[ 4500.715540] Lustre: DEBUG MARKER: == sanity test 56ba: test lfs find --component-end, -start, -count, and -flags ========================================================== 00:46:03 (1734396363)
[ 4503.367429] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ca: check lfs find --mirror-count\|-N and --mirror-state ========================================================== 00:46:06 \(1734396366\)
[ 4503.727880] Lustre: DEBUG MARKER: == sanity test 56ca: check lfs find --mirror-count|-N and --mirror-state ========================================================== 00:46:06 (1734396366)
[ 4506.098029] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56da: test lfs find with long paths ======= 00:46:09 \(1734396369\)
[ 4506.459971] Lustre: DEBUG MARKER: == sanity test 56da: test lfs find with long paths ======= 00:46:09 (1734396369)
[ 4514.079947] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56db: test \'lfs df -m\' only shows MDT devices ========================================================== 00:46:17 \(1734396377\)
[ 4514.442635] Lustre: DEBUG MARKER: == sanity test 56db: test 'lfs df -m' only shows MDT devices ========================================================== 00:46:17 (1734396377)
[ 4516.288264] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56dc: test \'lfs df -o\' only shows OST devices ========================================================== 00:46:19 \(1734396379\)
[ 4516.648543] Lustre: DEBUG MARKER: == sanity test 56dc: test 'lfs df -o' only shows OST devices ========================================================== 00:46:19 (1734396379)
[ 4518.484362] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56dd: test lfs find with mindepth argument ========================================================== 00:46:21 \(1734396381\)
[ 4518.859812] Lustre: DEBUG MARKER: == sanity test 56dd: test lfs find with mindepth argument ========================================================== 00:46:21 (1734396381)
[ 4520.690507] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ea: test lfs find -printf option ======== 00:46:24 \(1734396384\)
[ 4521.054329] Lustre: DEBUG MARKER: == sanity test 56ea: test lfs find -printf option ======== 00:46:24 (1734396384)
[ 4523.380721] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56ea 		2>/dev/null || echo foo
[ 4525.731906] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56ea |
[ 4525.731906] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID  | sort -u | tr '\n' ' ' 
[ 4546.678464] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_56ea 		2>/dev/null || echo foo
[ 4547.166860] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eaa: test lfs find -printf added functions ========================================================== 00:46:50 \(1734396410\)
[ 4547.620028] Lustre: DEBUG MARKER: == sanity test 56eaa: test lfs find -printf added functions ========================================================== 00:46:50 (1734396410)
[ 4557.466869] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eab: test lfs find -ls function ========= 00:47:00 \(1734396420\)
[ 4557.832677] Lustre: DEBUG MARKER: == sanity test 56eab: test lfs find -ls function ========= 00:47:00 (1734396420)
[ 4560.684931] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eb: check lfs getstripe on symlink ====== 00:47:04 \(1734396424\)
[ 4561.049309] Lustre: DEBUG MARKER: == sanity test 56eb: check lfs getstripe on symlink ====== 00:47:04 (1734396424)
[ 4562.926479] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ebb: check \/usr\/bin\/lfs getdirstripe for FIFO file ========================================================== 00:47:06 \(1734396426\)
[ 4563.293534] Lustre: DEBUG MARKER: == sanity test 56ebb: check /usr/bin/lfs getdirstripe for FIFO file ========================================================== 00:47:06 (1734396426)
[ 4565.110555] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ec: check lfs getstripe,setstripe --hex --yaml ========================================================== 00:47:08 \(1734396428\)
[ 4565.475605] Lustre: DEBUG MARKER: == sanity test 56ec: check lfs getstripe,setstripe --hex --yaml ========================================================== 00:47:08 (1734396428)
[ 4572.372285] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ed: verify new YAML format is valid and back-compatible ========================================================== 00:47:15 \(1734396435\)
[ 4572.741676] Lustre: DEBUG MARKER: == sanity test 56ed: verify new YAML format is valid and back-compatible ========================================================== 00:47:15 (1734396435)
[ 4576.764528] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eda: check lfs find --links ============= 00:47:20 \(1734396440\)
[ 4577.128546] Lustre: DEBUG MARKER: == sanity test 56eda: check lfs find --links ============= 00:47:20 (1734396440)
[ 4579.008457] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56edb: check lfs find --links for directory striped on multiple MDTs ========================================================== 00:47:22 \(1734396442\)
[ 4579.373444] Lustre: DEBUG MARKER: == sanity test 56edb: check lfs find --links for directory striped on multiple MDTs ========================================================== 00:47:22 (1734396442)
[ 4579.788448] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_56edb needs \>= 2 MDTs
[ 4580.152538] Lustre: DEBUG MARKER: SKIP: sanity test_56edb needs >= 2 MDTs
[ 4580.593827] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ef: lfs find with multiple paths ======== 00:47:24 \(1734396444\)
[ 4580.959001] Lustre: DEBUG MARKER: == sanity test 56ef: lfs find with multiple paths ======== 00:47:24 (1734396444)
[ 4582.863266] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eg: lfs find -xattr ===================== 00:47:26 \(1734396446\)
[ 4583.228367] Lustre: DEBUG MARKER: == sanity test 56eg: lfs find -xattr ===================== 00:47:26 (1734396446)
[ 4585.181996] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56eh: check lfs find --skip =============== 00:47:28 \(1734396448\)
[ 4585.548480] Lustre: DEBUG MARKER: == sanity test 56eh: check lfs find --skip =============== 00:47:28 (1734396448)
[ 4588.048096] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ei: test lfs find --printf prints correct projid for special files ========================================================== 00:47:31 \(1734396451\)
[ 4593.417951] Lustre: DEBUG MARKER: == sanity test 56ei: test lfs find --printf prints correct projid for special files ========================================================== 00:47:31 (1734396451)
[ 4600.324641] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 56ej: lfs migration --non-block copy ====== 00:47:43 \(1734396463\)
[ 4600.694344] Lustre: DEBUG MARKER: == sanity test 56ej: lfs migration --non-block copy ====== 00:47:43 (1734396463)
[ 4602.810087] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 57a: verify MDS filesystem created with large inodes ============================================================ 00:47:46 \(1734396466\)
[ 4603.187921] Lustre: DEBUG MARKER: == sanity test 57a: verify MDS filesystem created with large inodes ============================================================ 00:47:46 (1734396466)
[ 4605.945278] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 57b: default LOV EAs are stored inside large inodes ============================================================= 00:47:49 \(1734396469\)
[ 4606.310227] Lustre: DEBUG MARKER: == sanity test 57b: default LOV EAs are stored inside large inodes ============================================================= 00:47:49 (1734396469)
[ 4612.147889] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 58: verify cross-platform wire constants ======================================================================== 00:47:55 \(1734396475\)
[ 4612.513188] Lustre: DEBUG MARKER: == sanity test 58: verify cross-platform wire constants ======================================================================== 00:47:55 (1734396475)
[ 4614.320629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 59: verify cancellation of llog records async =================================================================== 00:47:57 \(1734396477\)
[ 4619.688163] Lustre: DEBUG MARKER: == sanity test 59: verify cancellation of llog records async =================================================================== 00:47:57 (1734396477)
[ 4620.364768] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4621.179940] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4622.607890] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4623.387038] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4638.787682] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60a: llog_test run from kernel module and test llog_reader ========================================================== 00:48:22 \(1734396502\)
[ 4639.152347] Lustre: DEBUG MARKER: == sanity test 60a: llog_test run from kernel module and test llog_reader ========================================================== 00:48:22 (1734396502)
[ 4639.876241] Lustre: DEBUG MARKER: /usr/sbin/lctl mark test_60 run 731 - from kernel mode
[ 4640.239538] Lustre: DEBUG MARKER: test_60 run 731 - from kernel mode
[ 4694.207011] Autotest: Test running for 75 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 4717.965120] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[ 4717.965128] Lustre: lustre-MDT0000-mdc-ffff988c12221000: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 4717.973970] Lustre: Skipped 3 previous similar messages
[ 4717.975892] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28ee113025 to 0xdf05ec28ef271a42
[ 4717.978575] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[ 4717.980376] Lustre: Skipped 1 previous similar message
[ 4717.985667] LustreError: 4379:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988c624a6700 x1818642152316416/t12885747318(12885747318) o101->lustre-MDT0000-mdc-ffff988c12221000@10.240.22.189@tcp:12/10 lens 4616/608 e 0 to 0 dl 1734396597 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'lfs.0' uid:0 gid:0
[ 4717.990758] LustreError: 4379:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 1 previous similar message
[ 4718.454136] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60b: limit repeated messages from CERROR\/CWARN ========================================================== 00:49:41 \(1734396581\)
[ 4718.819339] Lustre: DEBUG MARKER: == sanity test 60b: limit repeated messages from CERROR/CWARN ========================================================== 00:49:41 (1734396581)
[ 4721.449179] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60c: unlink file when mds full ============ 00:49:44 \(1734396584\)
[ 4721.817645] Lustre: DEBUG MARKER: == sanity test 60c: unlink file when mds full ============ 00:49:44 (1734396584)
[ 4722.489898] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4724.108053] Lustre: 4381:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734396571/real 1734396571]  req@ffff988c718a6a40 x1818642160366464/t0(0) o400->lustre-MDT0000-mdc-ffff988c12221000@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734396587 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 4726.685789] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4728.103323] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 4728.204035] Lustre: 4380:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734396576/real 1734396576]  req@ffff988c718a5040 x1818642160367616/t0(0) o400->lustre-MDT0000-mdc-ffff988c12221000@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734396592 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[ 4731.209988] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4733.747214] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60d: test printk console message masking == 00:49:57 \(1734396597\)
[ 4734.110816] Lustre: DEBUG MARKER: == sanity test 60d: test printk console message masking == 00:49:57 (1734396597)
[ 4734.121260] Lustre: DEBUG MARKER: test message ID 8347 15641
[ 4736.202330] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60e: no space while new llog is being created ========================================================== 00:49:59 \(1734396599\)
[ 4736.567743] Lustre: DEBUG MARKER: == sanity test 60e: no space while new llog is being created ========================================================== 00:49:59 (1734396599)
[ 4738.778510] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60f: change debug_path works ============== 00:50:02 \(1734396602\)
[ 4739.137575] Lustre: DEBUG MARKER: == sanity test 60f: change debug_path works ============== 00:50:02 (1734396602)
[ 4739.180896] LustreError: dumping log to /tmp/f60f.sanity.1734396603.248888
[ 4741.852970] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60g: transaction abort won\'t cause MDT hung ========================================================== 00:50:05 \(1734396605\)
[ 4742.212905] Lustre: DEBUG MARKER: == sanity test 60g: transaction abort won't cause MDT hung ========================================================== 00:50:05 (1734396605)
[ 4778.612614] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60h: striped directory with missing stripes can be accessed ========================================================== 00:50:42 \(1734396642\)
[ 4778.980314] Lustre: DEBUG MARKER: == sanity test 60h: striped directory with missing stripes can be accessed ========================================================== 00:50:42 (1734396642)
[ 4779.399266] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_60h Need at least 2 MDTs
[ 4779.764647] Lustre: DEBUG MARKER: SKIP: sanity test_60h Need at least 2 MDTs
[ 4780.193265] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_60i skipping SLOW test 60i
[ 4780.558658] Lustre: DEBUG MARKER: SKIP: sanity test_60i skipping SLOW test 60i
[ 4780.995215] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 60j: llog_reader reports corruptions ====== 00:50:44 \(1734396644\)
[ 4781.357348] Lustre: DEBUG MARKER: == sanity test 60j: llog_reader reports corruptions ====== 00:50:44 (1734396644)
[ 4789.370151] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 61a: mmap\(\) writes don\'t make sync hang ========================================================================== 00:50:52 \(1734396652\)
[ 4789.734110] Lustre: DEBUG MARKER: == sanity test 61a: mmap() writes don't make sync hang ========================================================================== 00:50:52 (1734396652)
[ 4792.160740] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 61b: mmap\(\) of unstriped file is successful ========================================================== 00:50:55 \(1734396655\)
[ 4792.522433] Lustre: DEBUG MARKER: == sanity test 61b: mmap() of unstriped file is successful ========================================================== 00:50:55 (1734396655)
[ 4794.330133] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 63a: Verify oig_wait interruption does not crash ================================================================= 00:50:57 \(1734396657\)
[ 4794.695188] Lustre: DEBUG MARKER: == sanity test 63a: Verify oig_wait interruption does not crash ================================================================= 00:50:57 (1734396657)
[ 4855.997676] Lustre: DEBUG MARKER: sync
[ 4858.091973] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 4858.810384] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 4860.914582] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 63b: async write errors should be returned to fsync ============================================================= 00:52:04 \(1734396724\)
[ 4861.283267] Lustre: DEBUG MARKER: == sanity test 63b: async write errors should be returned to fsync ============================================================= 00:52:04 (1734396724)
[ 4862.097619] Lustre: *** cfs_fail_loc=406, val=0***
[ 4862.098698] LustreError: 260373:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0005-osc-ffff988c12221000: prep_req failed: rc = -12
[ 4862.100949] LustreError: 260373:0:(osc_cache.c:2129:osc_check_rpcs()) Write request failed with -12
[ 4864.662002] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 4866.215145] Lustre: DEBUG MARKER: sync
[ 4867.888546] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 4868.584947] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 4870.687252] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64a: verify filter grant calculations \(in kernel\) =============================================================== 00:52:14 \(1734396734\)
[ 4871.056766] Lustre: DEBUG MARKER: == sanity test 64a: verify filter grant calculations (in kernel) =============================================================== 00:52:14 (1734396734)
[ 4871.886730] Lustre: DEBUG MARKER: sync
[ 4873.600877] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 4874.291561] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 4876.384489] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_64b skipping SLOW test 64b
[ 4876.743025] Lustre: DEBUG MARKER: SKIP: sanity test_64b skipping SLOW test 64b
[ 4877.178218] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64c: verify grant shrink ================== 00:52:20 \(1734396740\)
[ 4877.540321] Lustre: DEBUG MARKER: == sanity test 64c: verify grant shrink ================== 00:52:20 (1734396740)
[ 4878.355165] Lustre: DEBUG MARKER: sync
[ 4880.337019] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 4881.036189] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 4883.111993] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64d: check grant limit exceed ============= 00:52:26 \(1734396746\)
[ 4883.477711] Lustre: DEBUG MARKER: == sanity test 64d: check grant limit exceed ============= 00:52:26 (1734396746)
[ 4900.554519] Lustre: DEBUG MARKER: sync
[ 4905.455252] Lustre: DEBUG MARKER: /usr/bin/lfs df
[ 4906.179241] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre-*.cur_*grant_bytes
[ 4916.531533] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64e: check grant consumption \(no grant allocation\) ========================================================== 00:52:59 \(1734396779\)
[ 4916.892394] Lustre: DEBUG MARKER: == sanity test 64e: check grant consumption (no grant allocation) ========================================================== 00:52:59 (1734396779)
[ 4916.919847] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 4916.932027] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 4917.103081] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 4917.556727] Lustre: Unmounted lustre-client
[ 4917.567341] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 4917.580982] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 4917.627482] Lustre: Mounted lustre-client
[ 4918.491935] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 4918.504527] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 4918.679923] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 4918.728301] Lustre: Unmounted lustre-client
[ 4918.737230] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 4918.748595] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 4918.795836] Lustre: Mounted lustre-client
[ 4921.661653] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64f: check grant consumption \(with grant allocation\) ========================================================== 00:53:05 \(1734396785\)
[ 4922.031455] Lustre: DEBUG MARKER: == sanity test 64f: check grant consumption (with grant allocation) ========================================================== 00:53:05 (1734396785)
[ 4922.050678] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 4922.062590] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 4922.251627] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 4922.295736] Lustre: Unmounted lustre-client
[ 4922.305142] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 4922.317260] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 4922.367126] Lustre: Mounted lustre-client
[ 4922.631624] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 4922.644073] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 4922.829062] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 4922.938087] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 4922.949186] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 4925.396631] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64g: grant shrink on MDT ================== 00:53:08 \(1734396788\)
[ 4925.755733] Lustre: DEBUG MARKER: == sanity test 64g: grant shrink on MDT ================== 00:53:08 (1734396788)
[ 4995.001352] Autotest: Test running for 80 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5091.004892] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64h: grant shrink on read ================= 00:55:54 \(1734396954\)
[ 5091.366671] Lustre: DEBUG MARKER: == sanity test 64h: grant shrink on read ================= 00:55:54 (1734396954)
[ 5091.710294] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param osc.*OST0000*.grant_shrink);
[ 5091.710294] 			 [[ -z "" ]] && param= ||
[ 5091.710294] 			 param=$(grep  <<< "$params");
[ 5091.710294] 			 [[ -z $param ]] && param="$params";
[ 5091.710294] 			 while read s; do echo client $s;
[ 5091.710294] 			 done <<< "$param"
[ 5091.730369] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param osc.*OST0000*.grant_shrink_interval);
[ 5091.730369] 			 [[ -z "" ]] && param= ||
[ 5091.730369] 			 param=$(grep  <<< "$params");
[ 5091.730369] 			 [[ -z $param ]] && param="$params";
[ 5091.730369] 			 while read s; do echo client $s;
[ 5091.730369] 			 done <<< "$param"
[ 5102.136865] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n osc.lustre-OST0000-osc-ffff988c74e25800.grant_shrink=1
[ 5102.152483] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n osc.lustre-OST0000-osc-ffff988c74e25800.grant_shrink_interval=1200
[ 5102.593584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 64i: shrink on reconnect ================== 00:56:06 \(1734396966\)
[ 5102.954216] Lustre: DEBUG MARKER: == sanity test 64i: shrink on reconnect ================== 00:56:06 (1734396966)
[ 5107.338490] Lustre: lustre-OST0000-osc-ffff988c74e25800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 5120.329720] Lustre: 4380:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734396968/real 1734396968]  req@ffff988c0eee30c0 x1818642169120640/t0(0) o17->lustre-OST0000-osc-ffff988c74e25800@10.240.22.182@tcp:28/4 lens 456/432 e 0 to 1 dl 1734396984 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'lctl.0' uid:0 gid:0
[ 5120.625373] Lustre: lustre-OST0000-osc-ffff988c74e25800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[ 5120.627460] Lustre: Skipped 1 previous similar message
[ 5123.844437] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5125.112412] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid
[ 5125.672920] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid
[ 5125.875410] Lustre: DEBUG MARKER: lctl get_param -n at_max
[ 5126.271853] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-[-0-9a-f]\*.ost_server_uuid in FULL state after 0 sec
[ 5126.773590] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec
[ 5130.684538] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65a: directory with no stripe info ======== 00:56:34 \(1734396994\)
[ 5131.046105] Lustre: DEBUG MARKER: == sanity test 65a: directory with no stripe info ======== 00:56:34 (1734396994)
[ 5132.873733] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65b: directory setstripe -S stripe_size\*2 -i 0 -c 1 ========================================================== 00:56:36 \(1734396996\)
[ 5133.234399] Lustre: DEBUG MARKER: == sanity test 65b: directory setstripe -S stripe_size*2 -i 0 -c 1 ========================================================== 00:56:36 (1734396996)
[ 5135.075689] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65c: directory setstripe -S stripe_size\*4 -i 1 -c 6 ========================================================== 00:56:38 \(1734396998\)
[ 5135.434574] Lustre: DEBUG MARKER: == sanity test 65c: directory setstripe -S stripe_size*4 -i 1 -c 6 ========================================================== 00:56:38 (1734396998)
[ 5137.286664] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65d: directory setstripe -S stripe_size -c stripe_count ========================================================== 00:56:40 \(1734397000\)
[ 5137.648852] Lustre: DEBUG MARKER: == sanity test 65d: directory setstripe -S stripe_size -c stripe_count ========================================================== 00:56:40 (1734397000)
[ 5139.511586] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65e: directory setstripe defaults ========= 00:56:42 \(1734397002\)
[ 5139.870562] Lustre: DEBUG MARKER: == sanity test 65e: directory setstripe defaults ========= 00:56:42 (1734397002)
[ 5141.773203] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65f: dir setstripe permission \(should return error\) ============================================================= 00:56:45 \(1734397005\)
[ 5142.144157] Lustre: DEBUG MARKER: == sanity test 65f: dir setstripe permission (should return error) ============================================================= 00:56:45 (1734397005)
[ 5143.966789] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65g: directory setstripe -d =============== 00:56:47 \(1734397007\)
[ 5144.331273] Lustre: DEBUG MARKER: == sanity test 65g: directory setstripe -d =============== 00:56:47 (1734397007)
[ 5146.191381] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65h: directory stripe info inherit ============================================================================== 00:56:49 \(1734397009\)
[ 5146.551005] Lustre: DEBUG MARKER: == sanity test 65h: directory stripe info inherit ============================================================================== 00:56:49 (1734397009)
[ 5148.430034] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65i: various tests to set root directory striping ========================================================== 00:56:51 \(1734397011\)
[ 5148.793454] Lustre: DEBUG MARKER: == sanity test 65i: various tests to set root directory striping ========================================================== 00:56:51 (1734397011)
[ 5155.409073] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65j: set default striping on root directory \(bug 6367\)=========================================================== 00:56:58 \(1734397018\)
[ 5155.772472] Lustre: DEBUG MARKER: == sanity test 65j: set default striping on root directory (bug 6367)=========================================================== 00:56:58 (1734397018)
[ 5159.101959] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65k: validate manual striping works properly with deactivated OSCs ========================================================== 00:57:02 \(1734397022\)
[ 5159.464633] Lustre: DEBUG MARKER: == sanity test 65k: validate manual striping works properly with deactivated OSCs ========================================================== 00:57:02 (1734397022)
[ 5162.935586] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5164.170705] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5166.916878] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5167.859497] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5173.590811] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5174.786622] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5177.576504] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5178.545484] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5189.204767] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5190.410731] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5193.153109] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5194.088671] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5199.754196] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5200.936455] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5203.679781] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5204.631222] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5210.331488] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5211.515937] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5214.290053] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5215.246445] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5220.931924] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5222.136681] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5224.866196] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5225.805112] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5231.492068] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5232.654542] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5235.398166] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 5236.361511] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 5242.813016] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65l: lfs find on -1 stripe dir ================================================================================== 00:58:26 \(1734397106\)
[ 5243.174864] Lustre: DEBUG MARKER: == sanity test 65l: lfs find on -1 stripe dir ================================================================================== 00:58:26 (1734397106)
[ 5245.028090] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65m: normal user can\'t set filesystem default stripe ========================================================== 00:58:28 \(1734397108\)
[ 5245.396633] Lustre: DEBUG MARKER: == sanity test 65m: normal user can't set filesystem default stripe ========================================================== 00:58:28 (1734397108)
[ 5247.202498] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65n: don\'t inherit default layout from root for new subdirectories ========================================================== 00:58:30 \(1734397110\)
[ 5247.562923] Lustre: DEBUG MARKER: == sanity test 65n: don't inherit default layout from root for new subdirectories ========================================================== 00:58:30 (1734397110)
[ 5249.985307] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65n 		2>/dev/null || echo foo
[ 5252.337952] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65n |
[ 5252.337952] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID  | sort -u | tr '\n' ' ' 
[ 5252.874713] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_65n needs \>= 2 MDTs
[ 5253.256571] Lustre: DEBUG MARKER: SKIP: sanity test_65n needs >= 2 MDTs
[ 5272.674903] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65n 		2>/dev/null || echo foo
[ 5273.135349] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65o: pool inheritance for mdt component === 00:58:56 \(1734397136\)
[ 5273.582086] Lustre: DEBUG MARKER: == sanity test 65o: pool inheritance for mdt component === 00:58:56 (1734397136)
[ 5275.910281] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65o 		2>/dev/null || echo foo
[ 5278.264695] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65o |
[ 5278.264695] 			grep -e lustre-OST0000_UUID -e lustre-OST0001_UUID -e lustre-OST0002_UUID -e lustre-OST0003_UUID -e lustre-OST0004_UUID -e lustre-OST0005_UUID -e lustre-OST0006_UUID  | sort -u | tr '\n' ' ' 
[ 5296.295033] Autotest: Test running for 85 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5297.235096] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_65o 		2>/dev/null || echo foo
[ 5297.684379] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65p: setstripe with yaml file and huge number ========================================================== 00:59:21 \(1734397161\)
[ 5298.061059] Lustre: DEBUG MARKER: == sanity test 65p: setstripe with yaml file and huge number ========================================================== 00:59:21 (1734397161)
[ 5299.965720] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65q: setstripe with \>=8E offset should fail ========================================================== 00:59:23 \(1734397163\)
[ 5300.333628] Lustre: DEBUG MARKER: == sanity test 65q: setstripe with >=8E offset should fail ========================================================== 00:59:23 (1734397163)
[ 5302.232790] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 65r: prevent all-zero offsets ============= 00:59:25 \(1734397165\)
[ 5302.595705] Lustre: DEBUG MARKER: == sanity test 65r: prevent all-zero offsets ============= 00:59:25 (1734397165)
[ 5304.497760] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 66: update inode blocks count on client ========================================================================= 00:59:27 \(1734397167\)
[ 5304.869712] Lustre: DEBUG MARKER: == sanity test 66: update inode blocks count on client ========================================================================= 00:59:27 (1734397167)
[ 5309.297086] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 69: verify oa2dentry return -ENOENT doesn\'t LBUG ================================================================ 00:59:32 \(1734397172\)
[ 5309.665723] Lustre: DEBUG MARKER: == sanity test 69: verify oa2dentry return -ENOENT doesn't LBUG ================================================================ 00:59:32 (1734397172)
[ 5313.505830] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 70a: verify health_check, health_write don\'t explode \(on OST\) ========================================================== 00:59:36 \(1734397176\)
[ 5313.871669] Lustre: DEBUG MARKER: == sanity test 70a: verify health_check, health_write don't explode (on OST) ========================================================== 00:59:36 (1734397176)
[ 5318.857255] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_71 skipping SLOW test 71
[ 5319.223577] Lustre: DEBUG MARKER: SKIP: sanity test_71 skipping SLOW test 71
[ 5324.666567] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 72a: Test that remove suid works properly \(bug5695\) ============================================================== 00:59:43 \(1734397183\)
[ 5325.036756] Lustre: DEBUG MARKER: == sanity test 72a: Test that remove suid works properly (bug5695) ============================================================== 00:59:43 (1734397183)
[ 5327.037323] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 72b: Test that we keep mode setting if without file data changed \(bug 24226\) ========================================================== 00:59:50 \(1734397190\)
[ 5327.418574] Lustre: DEBUG MARKER: == sanity test 72b: Test that we keep mode setting if without file data changed (bug 24226) ========================================================== 00:59:50 (1734397190)
[ 5329.380675] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 73: multiple MDC requests \(should not deadlock\) ========================================================== 00:59:52 \(1734397192\)
[ 5329.750950] Lustre: DEBUG MARKER: == sanity test 73: multiple MDC requests (should not deadlock) ========================================================== 00:59:52 (1734397192)
[ 5357.726727] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 74a: ldlm_enqueue freed-export error path, ls \(shouldn\'t LBUG\) ========================================================== 01:00:21 \(1734397221\)
[ 5358.095812] Lustre: DEBUG MARKER: == sanity test 74a: ldlm_enqueue freed-export error path, ls (shouldn't LBUG) ========================================================== 01:00:21 (1734397221)
[ 5360.023807] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 74b: ldlm_enqueue freed-export error path, touch \(shouldn\'t LBUG\) ========================================================== 01:00:23 \(1734397223\)
[ 5360.391952] Lustre: DEBUG MARKER: == sanity test 74b: ldlm_enqueue freed-export error path, touch (shouldn't LBUG) ========================================================== 01:00:23 (1734397223)
[ 5362.314314] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 74c: ldlm_lock_create error path, \(shouldn\'t LBUG\) ========================================================== 01:00:25 \(1734397225\)
[ 5362.682273] Lustre: DEBUG MARKER: == sanity test 74c: ldlm_lock_create error path, (shouldn't LBUG) ========================================================== 01:00:25 (1734397225)
[ 5362.698438] Lustre: *** cfs_fail_loc=319, val=0***
[ 5364.527302] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 76a: confirm clients recycle inodes properly ============================================================== 01:00:27 \(1734397227\)
[ 5364.893929] Lustre: DEBUG MARKER: == sanity test 76a: confirm clients recycle inodes properly ============================================================== 01:00:27 (1734397227)
[ 5370.218703] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 76b: confirm clients recycle directory inodes properly ============================================================== 01:00:33 \(1734397233\)
[ 5370.600008] Lustre: DEBUG MARKER: == sanity test 76b: confirm clients recycle directory inodes properly ============================================================== 01:00:33 (1734397233)
[ 5377.749655] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77a: normal checksum read\/write operation ========================================================== 01:00:41 \(1734397241\)
[ 5378.115936] Lustre: DEBUG MARKER: == sanity test 77a: normal checksum read/write operation ========================================================== 01:00:41 (1734397241)
[ 5380.226118] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77b: checksum error on client write, read ========================================================== 01:00:43 \(1734397243\)
[ 5380.586130] Lustre: DEBUG MARKER: == sanity test 77b: checksum error on client write, read ========================================================== 01:00:43 (1734397243)
[ 5380.705820] Lustre: *** cfs_fail_loc=409, val=0***
[ 5380.820157] LustreError: lustre-OST0003-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed on the client after we checksummed it - likely false positive due to mmap IO (bug 11742): from 10.240.22.182@tcp inode [0x200001b75:0x1ef3:0x0] object 0x300000bd0:1064 extent [0-4194303], original client csum 20a200de (type 20), server csum 20a200dd (type 20), client csum now 20a200dd
[ 5380.825906] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c10988000 x1818642176829824/t8589954515(8589954515) o4->lustre-OST0003-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/448 e 0 to 0 dl 1734397261 ref 3 fl Interpret:RQU/204/0 rc 0/0 job:'dd.0' uid:0 gid:0
[ 5382.480334] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32, rc = 0
[ 5382.847350] Lustre: DEBUG MARKER: set checksum type to crc32, rc = 0
[ 5382.918331] Lustre: *** cfs_fail_loc=408, val=0***
[ 5382.922900] LustreError: lustre-OST0003-osc-ffff988c74e25800: BAD READ CHECKSUM: from 10.240.22.182@tcp inode [0x200001b75:0x1ef3:0x0] object 0x300000bd0:1064 extent [0-4194303], client 727421c5/727421c5, server d61677fc, cksum_type 1
[ 5382.926537] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c10988000 x1818642176831616/t0(0) o3->lustre-OST0003-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 0 dl 1734397263 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5384.467640] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to adler, rc = 0
[ 5384.831063] Lustre: DEBUG MARKER: set checksum type to adler, rc = 0
[ 5384.909579] LustreError: lustre-OST0003-osc-ffff988c74e25800: BAD READ CHECKSUM: from 10.240.22.182@tcp inode [0x200001b75:0x1ef3:0x0] object 0x300000bd0:1064 extent [0-4194303], client fec0b317/fec0b317, server 51cfb3b6, cksum_type 2
[ 5384.913256] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c10988d00 x1818642176834304/t0(0) o3->lustre-OST0003-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 0 dl 1734397265 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5386.589493] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32c, rc = 0
[ 5386.951469] Lustre: DEBUG MARKER: set checksum type to crc32c, rc = 0
[ 5387.022004] Lustre: *** cfs_fail_loc=408, val=0***
[ 5387.023014] Lustre: Skipped 1 previous similar message
[ 5387.027396] LustreError: lustre-OST0003-osc-ffff988c74e25800: BAD READ CHECKSUM: from 10.240.22.182@tcp inode [0x200001b75:0x1ef3:0x0] object 0x300000bd0:1064 extent [0-4194303], client 1fb7e5fa/1fb7e5fa, server a0529b77, cksum_type 4
[ 5387.031078] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c308fe3c0 x1818642176835840/t0(0) o3->lustre-OST0003-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 0 dl 1734397267 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5388.704715] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 5389.069622] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 5389.139234] LustreError: lustre-OST0003-osc-ffff988c74e25800: BAD READ CHECKSUM: from 10.240.22.182@tcp inode [0x200001b75:0x1ef3:0x0] object 0x300000bd0:1064 extent [0-4194303], client 7d2ae018/7d2ae018, server bac2df79, cksum_type 10
[ 5390.812863] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5391.171354] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5391.237964] Lustre: *** cfs_fail_loc=408, val=0***
[ 5391.238945] Lustre: Skipped 1 previous similar message
[ 5391.241832] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c308ff740 x1818642176839040/t0(0) o3->lustre-OST0003-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 0 dl 1734397272 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5391.246869] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 1 previous similar message
[ 5397.912036] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 5398.275169] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 5398.344967] LustreError: lustre-OST0003-osc-ffff988c74e25800: BAD READ CHECKSUM: from 10.240.22.182@tcp inode [0x200001b75:0x1ef3:0x0] object 0x300000bd0:1064 extent [0-4194303], client 25cce880/25cce880, server e5c7e867, cksum_type 40
[ 5398.348661] LustreError: Skipped 1 previous similar message
[ 5399.840933] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 5400.201823] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 5400.268875] Lustre: *** cfs_fail_loc=408, val=0***
[ 5400.269893] Lustre: Skipped 1 previous similar message
[ 5400.272913] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c308fc340 x1818642176841856/t0(0) o3->lustre-OST0003-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 0 dl 1734397281 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'cmp.0' uid:0 gid:0
[ 5400.277951] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 1 previous similar message
[ 5401.876916] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5402.240923] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5404.097100] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77c: checksum error on client read with debug ========================================================== 01:01:07 \(1734397267\)
[ 5404.460749] Lustre: DEBUG MARKER: == sanity test 77c: checksum error on client read with debug ========================================================== 01:01:07 (1734397267)
[ 5405.787307] Lustre: 4380:0:(osc_request.c:2037:dump_all_bulk_pages()) /tmp/lustre-log-checksum_dump-osc-[0x200001b75:0x1ef4:0x0]:[0-1048575]-d8110568-981105c8: dumping checksum data
[ 5405.848907] LustreError: dumping log to /tmp/lustre-log.1734397269.4380
[ 5406.511247] LustreError: lustre-OST0000-osc-ffff988c74e25800: BAD READ CHECKSUM: from 10.240.22.182@tcp inode [0x200001b75:0x1ef4:0x0] object 0x2400013a0:2673 extent [0-1048575], client d8110568/d8110568, server 981105c8, cksum_type 20
[ 5406.515002] LustreError: Skipped 1 previous similar message
[ 5415.195607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77d: checksum error on OST direct write, read ========================================================== 01:01:18 \(1734397278\)
[ 5415.559238] Lustre: DEBUG MARKER: == sanity test 77d: checksum error on OST direct write, read ========================================================== 01:01:18 (1734397278)
[ 5415.891341] LustreError: lustre-OST0004-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed on the client after we checksummed it - likely false positive due to mmap IO (bug 11742): from 10.240.22.182@tcp inode [0x200001b75:0x1ef6:0x0] object 0x340000bd0:1710 extent [0-4194303], original client csum fec05011 (type 20), server csum fec05010 (type 20), client csum now fec05010
[ 5417.095860] Lustre: *** cfs_fail_loc=408, val=0***
[ 5417.096982] Lustre: Skipped 2 previous similar messages
[ 5417.099767] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c10989040 x1818642176852608/t0(0) o3->lustre-OST0004-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 0 dl 1734397297 ref 2 fl Interpret:RMQU/200/0 rc 4194304/4194304 job:'directio.0' uid:0 gid:0
[ 5417.104850] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 2 previous similar messages
[ 5424.985914] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77f: repeat checksum error on write \(expect error\) ========================================================== 01:01:28 \(1734397288\)
[ 5425.350655] Lustre: DEBUG MARKER: == sanity test 77f: repeat checksum error on write (expect error) ========================================================== 01:01:28 (1734397288)
[ 5425.785924] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32, rc = 0
[ 5426.146269] Lustre: DEBUG MARKER: set checksum type to crc32, rc = 0
[ 5426.430156] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [4194304-8388607], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5430.003002] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [4194304-8388607], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5430.008130] LustreError: Skipped 3 previous similar messages
[ 5437.710731] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [4194304-8388607], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5437.715801] LustreError: Skipped 3 previous similar messages
[ 5449.586889] Lustre: *** cfs_fail_loc=409, val=0***
[ 5449.587982] Lustre: Skipped 26 previous similar messages
[ 5449.589039] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [0-4194303], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5449.593900] LustreError: Skipped 3 previous similar messages
[ 5449.595008] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c308fdd40 x1818642176859520/t8589955050(8589955050) o4->lustre-OST0005-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/448 e 0 to 0 dl 1734397329 ref 2 fl Interpret:RMQU/200/0 rc 0/0 job:'ptlrpcd_00_01.0' uid:0 gid:0
[ 5449.600235] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 12 previous similar messages
[ 5474.178805] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [4194304-8388607], original client csum 8b19b060 (type 1), server csum 8b19b05f (type 1), client csum now 8b19b060
[ 5474.183856] LustreError: Skipped 5 previous similar messages
[ 5485.362292] LustreError: 4381:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c74e25800: too many resent retries for object: 15032388560:1717: rc = -11
[ 5485.813360] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to adler, rc = 0
[ 5486.256546] Lustre: DEBUG MARKER: set checksum type to adler, rc = 0
[ 5508.041440] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [4194304-8388607], original client csum 7d33b9a0 (type 2), server csum 7d33b99f (type 2), client csum now 7d33b9a0
[ 5508.046574] LustreError: Skipped 15 previous similar messages
[ 5515.441599] Lustre: *** cfs_fail_loc=409, val=0***
[ 5515.442613] Lustre: Skipped 47 previous similar messages
[ 5515.443687] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c10988340 x1818642176868736/t8589955075(8589955075) o4->lustre-OST0005-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/448 e 0 to 0 dl 1734397394 ref 2 fl Interpret:RMQU/200/0 rc 0/0 job:'ptlrpcd_00_00.0' uid:0 gid:0
[ 5515.448890] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 21 previous similar messages
[ 5543.908279] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c74e25800: too many resent retries for object: 15032388560:1717: rc = -11
[ 5543.911082] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 5544.474781] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32c, rc = 0
[ 5544.926070] Lustre: DEBUG MARKER: set checksum type to crc32c, rc = 0
[ 5576.525129] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [4194304-8388607], original client csum de2bf73f (type 4), server csum de2bf73e (type 4), client csum now de2bf73f
[ 5576.532195] LustreError: Skipped 23 previous similar messages
[ 5597.140578] Autotest: Test running for 90 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5604.183243] LustreError: 4381:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c74e25800: too many resent retries for object: 15032388560:1717: rc = -11
[ 5604.186046] LustreError: 4381:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 5609.634729] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 5610.102024] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 5647.225803] Lustre: *** cfs_fail_loc=409, val=0***
[ 5647.226849] Lustre: Skipped 91 previous similar messages
[ 5647.227949] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -11  req@ffff988c10988d00 x1818642176886272/t8589955121(8589955121) o4->lustre-OST0005-osc-ffff988c74e25800@10.240.22.182@tcp:6/4 lens 488/448 e 0 to 0 dl 1734397526 ref 2 fl Interpret:RMQU/200/0 rc 0/0 job:'ptlrpcd_00_00.0' uid:0 gid:0
[ 5647.233238] LustreError: 4381:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 41 previous similar messages
[ 5666.600531] LustreError: 4381:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c74e25800: too many resent retries for object: 15032388560:1717: rc = -11
[ 5666.603354] LustreError: 4381:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 5667.043004] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5667.484659] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5706.544732] LustreError: lustre-OST0005-osc-ffff988c74e25800: BAD WRITE CHECKSUM: changed in transit before arrival at OST: from 10.240.22.182@tcp inode [0x200001b75:0x1ef7:0x0] object 0x380000bd0:1717 extent [0-4194303], original client csum fec05011 (type 20), server csum fec05010 (type 20), client csum now fec05011
[ 5706.549727] LustreError: Skipped 45 previous similar messages
[ 5726.070682] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c74e25800: too many resent retries for object: 15032388560:1717: rc = -11
[ 5726.073455] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 5726.513142] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 5726.952077] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 5785.396565] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c74e25800: too many resent retries for object: 15032388560:1717: rc = -11
[ 5785.399413] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 5785.887400] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 5786.342541] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 5843.805896] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c74e25800: too many resent retries for object: 15032388560:1717: rc = -11
[ 5843.808707] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 5844.244013] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5844.700207] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5846.537832] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77g: checksum error on OST write, read ==== 01:08:30 \(1734397710\)
[ 5846.901316] Lustre: DEBUG MARKER: == sanity test 77g: checksum error on OST write, read ==== 01:08:30 (1734397710)
[ 5849.004283] LustreError: lustre-OST0000-osc-ffff988c74e25800: BAD READ CHECKSUM: from 10.240.22.182@tcp inode [0x200001b75:0x1efa:0x0] object 0x2400013a0:2674 extent [0-4194303], client 20a200dd/20a200dd, server 1cba0a59, cksum_type 20
[ 5849.007966] LustreError: Skipped 1 previous similar message
[ 5852.060826] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77k: enable\/disable checksum correctly ==== 01:08:35 \(1734397715\)
[ 5852.418912] Lustre: DEBUG MARKER: == sanity test 77k: enable/disable checksum correctly ==== 01:08:35 (1734397715)
[ 5852.753358] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 5852.767474] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5852.779046] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5852.964232] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 5853.015365] Lustre: Unmounted lustre-client
[ 5853.016246] Lustre: Skipped 1 previous similar message
[ 5853.024965] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5853.036628] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 5853.090244] Lustre: Mounted lustre-client
[ 5853.091146] Lustre: Skipped 1 previous similar message
[ 5853.424643] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 5854.441409] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 5854.458505] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5854.469966] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5854.632936] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 5854.676965] Lustre: Unmounted lustre-client
[ 5854.684911] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5854.695519] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 5854.750030] Lustre: Mounted lustre-client
[ 5855.092172] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5855.104269] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5855.280308] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 5855.335590] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5855.346625] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,checksum 10.240.22.189@tcp:/lustre /mnt/lustre
[ 5855.425459] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5855.437112] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5855.615519] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 5855.670093] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5855.680998] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,nochecksum 10.240.22.189@tcp:/lustre /mnt/lustre
[ 5855.761975] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 5855.773664] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 5855.949962] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 5855.995981] Lustre: Unmounted lustre-client
[ 5855.996849] Lustre: Skipped 2 previous similar messages
[ 5856.004305] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 5856.015113] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 5856.063430] Lustre: Mounted lustre-client
[ 5856.064279] Lustre: Skipped 2 previous similar messages
[ 5857.754012] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 5858.773436] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 5859.789817] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n osc.lustre*.checksums | head -n1
[ 5860.223774] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77l: preferred checksum type is remembered after reconnected ========================================================== 01:08:43 \(1734397723\)
[ 5860.583878] Lustre: DEBUG MARKER: == sanity test 77l: preferred checksum type is remembered after reconnected ========================================================== 01:08:43 (1734397723)
[ 5861.019674] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to invalid, rc = 22
[ 5861.375962] Lustre: DEBUG MARKER: set checksum type to invalid, rc = 22
[ 5861.795918] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32, rc = 0
[ 5862.149219] Lustre: DEBUG MARKER: set checksum type to crc32, rc = 0
[ 5862.199459] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5862.226373] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5863.037500] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5863.388846] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5866.823650] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 3 sec
[ 5867.174342] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 3 sec
[ 5867.202672] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5867.230647] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5868.043341] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5868.391318] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5868.792172] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5869.141187] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5869.564760] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to adler, rc = 0
[ 5869.924961] Lustre: DEBUG MARKER: set checksum type to adler, rc = 0
[ 5869.977432] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5870.004731] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5870.829580] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5871.184915] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5882.708721] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 11 sec
[ 5883.167349] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 11 sec
[ 5883.191930] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5883.220192] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5884.039935] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5884.391751] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5884.796140] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5885.151336] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5885.577371] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to crc32c, rc = 0
[ 5885.939781] Lustre: DEBUG MARKER: set checksum type to crc32c, rc = 0
[ 5885.992795] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5886.020641] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5886.844746] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5887.199391] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5898.251674] Autotest: Test running for 95 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 5902.717026] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 10 sec
[ 5908.076563] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 10 sec
[ 5908.103410] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5908.132290] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5908.947365] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5909.304523] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5909.708483] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5910.059569] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5910.488268] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 5910.848812] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 5910.904463] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5910.932398] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5911.746870] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5912.099468] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5923.627373] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 11 sec
[ 5924.057985] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 11 sec
[ 5924.083262] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5924.112295] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5924.924905] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5925.278399] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5925.680897] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5926.033121] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5926.461279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5926.826114] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5926.876558] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5926.904577] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5927.715411] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5928.069524] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5938.589935] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 10 sec
[ 5939.024685] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 10 sec
[ 5939.047966] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5939.082776] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5939.900490] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5940.250475] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5940.659583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5941.010742] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5941.446019] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 5941.809338] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 5941.866204] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5941.895890] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5942.720490] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5943.073601] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5954.590633] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 11 sec
[ 5955.019024] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 11 sec
[ 5955.045747] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5955.074403] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5955.885119] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5956.235442] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5956.633379] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5956.981514] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5957.404231] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 5957.760313] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 5957.815975] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5957.844360] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5958.652192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5958.998757] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5969.512813] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 10 sec
[ 5969.958642] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in IDLE state after 10 sec
[ 5969.979806] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 5970.008579] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 5970.824531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5971.173367] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid 50
[ 5971.573941] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5971.925929] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c13bbd800.ost_server_uuid in FULL state after 0 sec
[ 5973.822909] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5974.180277] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5974.617408] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77m: Verify checksum_speed is correctly read ========================================================== 01:10:38 \(1734397838\)
[ 5974.973850] Lustre: DEBUG MARKER: == sanity test 77m: Verify checksum_speed is correctly read ========================================================== 01:10:38 (1734397838)
[ 5976.766941] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77n: Verify read from a hole inside contiguous blocks with T10PI ========================================================== 01:10:40 \(1734397840\)
[ 5977.128948] Lustre: DEBUG MARKER: == sanity test 77n: Verify read from a hole inside contiguous blocks with T10PI ========================================================== 01:10:40 (1734397840)
[ 5977.709580] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip512, rc = 0
[ 5978.066866] Lustre: DEBUG MARKER: set checksum type to t10ip512, rc = 0
[ 5978.483247] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5978.842579] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5979.256985] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc512, rc = 0
[ 5979.613730] Lustre: DEBUG MARKER: set checksum type to t10crc512, rc = 0
[ 5980.023554] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10crc4K, rc = 0
[ 5980.378692] Lustre: DEBUG MARKER: set checksum type to t10crc4K, rc = 0
[ 5982.163602] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[ 5982.524989] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[ 5982.958234] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 77o: Verify checksum_type for server \(mdt and ofd\(obdfilter\)\) ========================================================== 01:10:46 \(1734397846\)
[ 5983.316701] Lustre: DEBUG MARKER: == sanity test 77o: Verify checksum_type for server (mdt and ofd(obdfilter)) ========================================================== 01:10:46 (1734397846)
[ 5986.673950] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 78: handle large O_DIRECT writes correctly ====================================================================== 01:10:50 \(1734397850\)
[ 5987.031007] Lustre: DEBUG MARKER: == sanity test 78: handle large O_DIRECT writes correctly ====================================================================== 01:10:50 (1734397850)
[ 5990.185840] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 79: df report consistency check ================================================================================= 01:10:53 \(1734397853\)
[ 5990.546900] Lustre: DEBUG MARKER: == sanity test 79: df report consistency check ================================================================================= 01:10:53 (1734397853)
[ 6004.542740] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 80: Page eviction is equally fast at high offsets too ========================================================== 01:11:08 \(1734397868\)
[ 6004.907229] Lustre: DEBUG MARKER: == sanity test 80: Page eviction is equally fast at high offsets too ========================================================== 01:11:08 (1734397868)
[ 6008.399168] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 81a: OST should retry write when get -ENOSPC ========================================================================= 01:11:11 \(1734397871\)
[ 6008.759698] Lustre: DEBUG MARKER: == sanity test 81a: OST should retry write when get -ENOSPC ========================================================================= 01:11:11 (1734397871)
[ 6011.041415] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 81b: OST should return -ENOSPC when retry still fails ================================================================= 01:11:14 \(1734397874\)
[ 6011.401540] Lustre: DEBUG MARKER: == sanity test 81b: OST should return -ENOSPC when retry still fails ================================================================= 01:11:14 (1734397874)
[ 6013.754121] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 99: cvs strange file\/directory operations ========================================================== 01:11:17 \(1734397877\)
[ 6014.114138] Lustre: DEBUG MARKER: == sanity test 99: cvs strange file/directory operations ========================================================== 01:11:17 (1734397877)
[ 6025.226304] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 100: check local port using privileged port ========================================================== 01:11:23 \(1734397883\)
[ 6025.588192] Lustre: DEBUG MARKER: == sanity test 100: check local port using privileged port ========================================================== 01:11:23 (1734397883)
[ 6027.621813] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101a: check read-ahead for random reads === 01:11:31 \(1734397891\)
[ 6027.990807] Lustre: DEBUG MARKER: == sanity test 101a: check read-ahead for random reads === 01:11:31 (1734397891)
[ 6057.097325] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101b: check stride-io mode read-ahead =========================================================================== 01:12:00 \(1734397920\)
[ 6057.461228] Lustre: DEBUG MARKER: == sanity test 101b: check stride-io mode read-ahead =========================================================================== 01:12:00 (1734397920)
[ 6079.892576] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101c: check stripe_size aligned read-ahead ========================================================== 01:12:23 \(1734397943\)
[ 6080.261623] Lustre: DEBUG MARKER: == sanity test 101c: check stripe_size aligned read-ahead ========================================================== 01:12:23 (1734397943)
[ 6094.049008] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101d: file read with and without read-ahead enabled ========================================================== 01:12:37 \(1734397957\)
[ 6094.410828] Lustre: DEBUG MARKER: == sanity test 101d: file read with and without read-ahead enabled ========================================================== 01:12:37 (1734397957)
[ 6141.760466] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101e: check read-ahead for small read\(1k\) for small files\(500k\) ========================================================== 01:13:25 \(1734398005\)
[ 6142.123649] Lustre: DEBUG MARKER: == sanity test 101e: check read-ahead for small read(1k) for small files(500k) ========================================================== 01:13:25 (1734398005)
[ 6161.302116] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101f: check mmap read performance ========= 01:13:39 \(1734398019\)
[ 6166.665573] Lustre: DEBUG MARKER: == sanity test 101f: check mmap read performance ========= 01:13:39 (1734398019)
[ 6167.085155] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_101f no iozone installed
[ 6167.445944] Lustre: DEBUG MARKER: SKIP: sanity test_101f no iozone installed
[ 6167.885564] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101g: Big bulk\(4\/16 MiB\) readahead ======== 01:13:51 \(1734398031\)
[ 6168.249450] Lustre: DEBUG MARKER: == sanity test 101g: Big bulk(4/16 MiB) readahead ======== 01:13:51 (1734398031)
[ 6171.264439] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6171.278517] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6171.457608] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 6171.527469] Lustre: Unmounted lustre-client
[ 6171.537354] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6171.548472] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 6171.601006] Lustre: Mounted lustre-client
[ 6193.091492] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 6193.106215] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 6193.269787] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 6193.484162] Lustre: Unmounted lustre-client
[ 6193.492396] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 6193.503400] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 6193.555232] Lustre: Mounted lustre-client
[ 6195.510826] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101h: Readahead should cover current read window ========================================================== 01:14:18 \(1734398058\)
[ 6195.873190] Lustre: DEBUG MARKER: == sanity test 101h: Readahead should cover current read window ========================================================== 01:14:18 (1734398058)
[ 6199.227266] Autotest: Test running for 100 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 6200.026831] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101i: allow current readahead to exceed reservation ========================================================== 01:14:23 \(1734398063\)
[ 6200.392662] Lustre: DEBUG MARKER: == sanity test 101i: allow current readahead to exceed reservation ========================================================== 01:14:23 (1734398063)
[ 6202.931535] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101j: A complete read block should be submitted when no RA ========================================================== 01:14:26 \(1734398066\)
[ 6203.298199] Lustre: DEBUG MARKER: == sanity test 101j: A complete read block should be submitted when no RA ========================================================== 01:14:26 (1734398066)
[ 6210.905788] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 101m: read ahead for small file and last stripe of the file ========================================================== 01:14:34 \(1734398074\)
[ 6211.272274] Lustre: DEBUG MARKER: == sanity test 101m: read ahead for small file and last stripe of the file ========================================================== 01:14:34 (1734398074)
[ 6211.612123] Lustre: DEBUG MARKER: Test readahead: size=4096 ramax= iosz=1048576
[ 6211.710993] Lustre: DEBUG MARKER: Test readahead: size=16384 ramax= iosz=1048576
[ 6211.846999] Lustre: DEBUG MARKER: Test readahead: size=16385 ramax= iosz=1048576
[ 6211.905560] Lustre: DEBUG MARKER: Test readahead: size=16383 ramax= iosz=1048576
[ 6211.961791] Lustre: DEBUG MARKER: Test readahead: size=1048577 ramax= iosz=2097152
[ 6212.112928] Lustre: DEBUG MARKER: Test readahead: size=1064960 ramax= iosz=2097152
[ 6212.250446] Lustre: DEBUG MARKER: Test readahead: size=6307840 ramax= iosz=7340032
[ 6212.661675] Lustre: DEBUG MARKER: Test readahead: size=7356416 ramax= iosz=8388608
[ 6212.976848] Lustre: DEBUG MARKER: Test readahead: size=14696448 ramax= iosz=15728640
[ 6215.399472] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102a: user xattr test ============================================================================================ 01:14:38 \(1734398078\)
[ 6215.765110] Lustre: DEBUG MARKER: == sanity test 102a: user xattr test ============================================================================================ 01:14:38 (1734398078)
[ 6217.688791] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102b: getfattr\/setfattr for trusted.lov EAs ========================================================== 01:14:41 \(1734398081\)
[ 6218.047259] Lustre: DEBUG MARKER: == sanity test 102b: getfattr/setfattr for trusted.lov EAs ========================================================== 01:14:41 (1734398081)
[ 6221.044836] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102c: non-root getfattr\/setfattr for lustre.lov EAs ===================================================================== 01:14:44 \(1734398084\)
[ 6221.414641] Lustre: DEBUG MARKER: == sanity test 102c: non-root getfattr/setfattr for lustre.lov EAs ===================================================================== 01:14:44 (1734398084)
[ 6223.329150] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102d: tar restore stripe info from tarfile,not keep osts ========================================================== 01:14:46 \(1734398086\)
[ 6223.692439] Lustre: DEBUG MARKER: == sanity test 102d: tar restore stripe info from tarfile,not keep osts ========================================================== 01:14:46 (1734398086)
[ 6230.162729] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102f: tar copy files, not keep osts ======= 01:14:53 \(1734398093\)
[ 6230.522968] Lustre: DEBUG MARKER: == sanity test 102f: tar copy files, not keep osts ======= 01:14:53 (1734398093)
[ 6236.964404] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102h: grow xattr from inside inode to external block ========================================================== 01:15:00 \(1734398100\)
[ 6237.329947] Lustre: DEBUG MARKER: == sanity test 102h: grow xattr from inside inode to external block ========================================================== 01:15:00 (1734398100)
[ 6237.764490] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.big on \/mnt\/lustre\/f102h.sanity
[ 6238.124092] Lustre: DEBUG MARKER: save trusted.big on /mnt/lustre/f102h.sanity
[ 6238.549784] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.sml on \/mnt\/lustre\/f102h.sanity
[ 6238.910989] Lustre: DEBUG MARKER: save trusted.sml on /mnt/lustre/f102h.sanity
[ 6239.333427] Lustre: DEBUG MARKER: /usr/sbin/lctl mark grow trusted.sml on \/mnt\/lustre\/f102h.sanity
[ 6239.699430] Lustre: DEBUG MARKER: grow trusted.sml on /mnt/lustre/f102h.sanity
[ 6240.118961] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trusted.big still valid after growing trusted.sml
[ 6240.482090] Lustre: DEBUG MARKER: trusted.big still valid after growing trusted.sml
[ 6242.354051] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102ha: grow xattr from inside inode to external inode ========================================================== 01:15:05 \(1734398105\)
[ 6242.712348] Lustre: DEBUG MARKER: == sanity test 102ha: grow xattr from inside inode to external inode ========================================================== 01:15:05 (1734398105)
[ 6243.471124] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.big on \/mnt\/lustre\/f102ha.sanity
[ 6243.832024] Lustre: DEBUG MARKER: save trusted.big on /mnt/lustre/f102ha.sanity
[ 6244.263988] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.sml on \/mnt\/lustre\/f102ha.sanity
[ 6244.623838] Lustre: DEBUG MARKER: save trusted.sml on /mnt/lustre/f102ha.sanity
[ 6245.051594] Lustre: DEBUG MARKER: /usr/sbin/lctl mark grow trusted.sml on \/mnt\/lustre\/f102ha.sanity
[ 6245.411425] Lustre: DEBUG MARKER: grow trusted.sml on /mnt/lustre/f102ha.sanity
[ 6245.838468] Lustre: DEBUG MARKER: /usr/sbin/lctl mark trusted.big still valid after growing trusted.sml
[ 6246.198170] Lustre: DEBUG MARKER: trusted.big still valid after growing trusted.sml
[ 6246.645772] Lustre: DEBUG MARKER: /usr/sbin/lctl mark save trusted.big on \/mnt\/lustre\/f102ha.sanity
[ 6247.005647] Lustre: DEBUG MARKER: save trusted.big on /mnt/lustre/f102ha.sanity
[ 6248.869864] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102i: lgetxattr test on symbolic link ====================================================================== 01:15:12 \(1734398112\)
[ 6249.236215] Lustre: DEBUG MARKER: == sanity test 102i: lgetxattr test on symbolic link ====================================================================== 01:15:12 (1734398112)
[ 6251.100110] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102j: non-root tar restore stripe info from tarfile, not keep osts ============================================================= 01:15:14 \(1734398114\)
[ 6251.461014] Lustre: DEBUG MARKER: == sanity test 102j: non-root tar restore stripe info from tarfile, not keep osts ============================================================= 01:15:14 (1734398114)
[ 6257.661708] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102k: setfattr without parameter of value shouldn\'t cause a crash ========================================================== 01:15:21 \(1734398121\)
[ 6258.029473] Lustre: DEBUG MARKER: == sanity test 102k: setfattr without parameter of value shouldn't cause a crash ========================================================== 01:15:21 (1734398121)
[ 6259.944424] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102l: listxattr size test ============================================================================================ 01:15:23 \(1734398123\)
[ 6260.306404] Lustre: DEBUG MARKER: == sanity test 102l: listxattr size test ============================================================================================ 01:15:23 (1734398123)
[ 6262.133609] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102m: Ensure listxattr fails on small bufffer ================================================================== 01:15:25 \(1734398125\)
[ 6262.498921] Lustre: DEBUG MARKER: == sanity test 102m: Ensure listxattr fails on small bufffer ================================================================== 01:15:25 (1734398125)
[ 6264.342537] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102n: silently ignore setxattr on internal trusted xattrs ========================================================== 01:15:27 \(1734398127\)
[ 6264.705647] Lustre: DEBUG MARKER: == sanity test 102n: silently ignore setxattr on internal trusted xattrs ========================================================== 01:15:27 (1734398127)
[ 6266.705792] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102p: check setxattr\(2\) correctly fails without permission ========================================================== 01:15:30 \(1734398130\)
[ 6267.066725] Lustre: DEBUG MARKER: == sanity test 102p: check setxattr(2) correctly fails without permission ========================================================== 01:15:30 (1734398130)
[ 6268.939246] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102q: flistxattr should not return trusted.link EAs for orphans ========================================================== 01:15:32 \(1734398132\)
[ 6269.302834] Lustre: DEBUG MARKER: == sanity test 102q: flistxattr should not return trusted.link EAs for orphans ========================================================== 01:15:32 (1734398132)
[ 6271.157662] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102r: set EAs with empty values =========== 01:15:34 \(1734398134\)
[ 6271.519844] Lustre: DEBUG MARKER: == sanity test 102r: set EAs with empty values =========== 01:15:34 (1734398134)
[ 6273.469180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102s: getting nonexistent xattrs should fail ========================================================== 01:15:36 \(1734398136\)
[ 6273.831825] Lustre: DEBUG MARKER: == sanity test 102s: getting nonexistent xattrs should fail ========================================================== 01:15:36 (1734398136)
[ 6273.852178] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[ 6273.852178] 			 [[ -z "" ]] && param= ||
[ 6273.852178] 			 param=$(grep  <<< "$params");
[ 6273.852178] 			 [[ -z $param ]] && param="$params";
[ 6273.852178] 			 while read s; do echo client $s;
[ 6273.852178] 			 done <<< "$param"
[ 6273.885073] LustreError: lustre-MDT0000-mdc-ffff988c12f63800: operation mds_getxattr to node 10.240.22.189@tcp failed: rc = -95
[ 6273.929727] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff988c12f63800.xattr_cache=1
[ 6275.759920] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 102t: zero length xattr values handled correctly ========================================================== 01:15:39 \(1734398139\)
[ 6276.119321] Lustre: DEBUG MARKER: == sanity test 102t: zero length xattr values handled correctly ========================================================== 01:15:39 (1734398139)
[ 6276.139558] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[ 6276.139558] 			 [[ -z "" ]] && param= ||
[ 6276.139558] 			 param=$(grep  <<< "$params");
[ 6276.139558] 			 [[ -z $param ]] && param="$params";
[ 6276.139558] 			 while read s; do echo client $s;
[ 6276.139558] 			 done <<< "$param"
[ 6276.252147] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff988c12f63800.xattr_cache=1
[ 6278.069194] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103a: acl test ============================ 01:15:41 \(1734398141\)
[ 6278.427881] Lustre: DEBUG MARKER: == sanity test 103a: acl test ============================ 01:15:41 (1734398141)
[ 6307.643013] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103b: umask lfs setstripe ================= 01:16:11 \(1734398171\)
[ 6308.009122] Lustre: DEBUG MARKER: == sanity test 103b: umask lfs setstripe ================= 01:16:11 (1734398171)
[ 6329.316607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103c: \'cp -rp\' won\'t set empty acl ======== 01:16:32 \(1734398192\)
[ 6329.743885] Lustre: DEBUG MARKER: == sanity test 103c: 'cp -rp' won't set empty acl ======== 01:16:32 (1734398192)
[ 6331.846551] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103e: inheritance of big amount of default ACLs ========================================================== 01:16:35 \(1734398195\)
[ 6332.208298] Lustre: DEBUG MARKER: == sanity test 103e: inheritance of big amount of default ACLs ========================================================== 01:16:35 (1734398195)
[ 6444.109808] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 103f: changelog doesn\'t interfere with default ACLs buffers ========================================================== 01:18:27 \(1734398307\)
[ 6444.479784] Lustre: DEBUG MARKER: == sanity test 103f: changelog doesn't interfere with default ACLs buffers ========================================================== 01:18:27 (1734398307)
[ 6448.561727] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104a: lfs df [-ih] [path] test =================================================================================== 01:18:32 \(1734398312\)
[ 6448.926290] Lustre: DEBUG MARKER: == sanity test 104a: lfs df [-ih] [path] test =================================================================================== 01:18:32 (1734398312)
[ 6448.988339] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request
[ 6449.005561] Lustre: lustre-OST0000-osc-ffff988c12f63800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 6449.012524] LustreError: lustre-OST0000-osc-ffff988c12f63800: This client was evicted by lustre-OST0000; in progress operations using this service will fail.
[ 6449.015712] Lustre: lustre-OST0000-osc-ffff988c12f63800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[ 6449.023616] Lustre: DEBUG MARKER: lctl get_param -n at_min
[ 6449.051794] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 6449.878593] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state \(FULL\|IDLE\) osc.lustre-OST0000-osc-ffff988c12f63800.ost_server_uuid 50
[ 6450.229434] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff988c12f63800.ost_server_uuid 50
[ 6450.630909] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c12f63800.ost_server_uuid in FULL state after 0 sec
[ 6450.979353] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c12f63800.ost_server_uuid in FULL state after 0 sec
[ 6452.913123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104b: runas -u 500 -g 500 lfs check servers test ============================================================================== 01:18:36 \(1734398316\)
[ 6453.277890] Lustre: DEBUG MARKER: == sanity test 104b: runas -u 500 -g 500 lfs check servers test ============================================================================== 01:18:36 (1734398316)
[ 6455.133575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104c: Verify df vs lfs_df stays same after recordsize change ========================================================== 01:18:38 \(1734398318\)
[ 6455.492209] Lustre: DEBUG MARKER: == sanity test 104c: Verify df vs lfs_df stays same after recordsize change ========================================================== 01:18:38 (1734398318)
[ 6455.905782] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_104c zfs only test
[ 6456.267616] Lustre: DEBUG MARKER: SKIP: sanity test_104c zfs only test
[ 6456.771532] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 104d: runas -u 500 -g 500 lctl dl test ==== 01:18:40 \(1734398320\)
[ 6457.132305] Lustre: DEBUG MARKER: == sanity test 104d: runas -u 500 -g 500 lctl dl test ==== 01:18:40 (1734398320)
[ 6459.023659] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105a: flock when mounted without -o flock test ================================================================== 01:18:42 \(1734398322\)
[ 6459.386901] Lustre: DEBUG MARKER: == sanity test 105a: flock when mounted without -o flock test ================================================================== 01:18:42 (1734398322)
[ 6461.269216] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105b: fcntl when mounted without -o flock test ================================================================== 01:18:44 \(1734398324\)
[ 6461.629804] Lustre: DEBUG MARKER: == sanity test 105b: fcntl when mounted without -o flock test ================================================================== 01:18:44 (1734398324)
[ 6463.493095] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105c: lockf when mounted without -o flock test ========================================================== 01:18:46 \(1734398326\)
[ 6463.854909] Lustre: DEBUG MARKER: == sanity test 105c: lockf when mounted without -o flock test ========================================================== 01:18:46 (1734398326)
[ 6465.727707] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105d: flock race \(should not freeze\) ================================================================== 01:18:49 \(1734398329\)
[ 6466.090017] Lustre: DEBUG MARKER: == sanity test 105d: flock race (should not freeze) ================================================================== 01:18:49 (1734398329)
[ 6466.128723] LustreError: 352136:0:(ldlm_flock.c:850:ldlm_flock_completion_ast()) cfs_fail_timeout id 315 sleeping for 10000ms
[ 6476.217697] LustreError: 352136:0:(ldlm_flock.c:850:ldlm_flock_completion_ast()) cfs_fail_timeout id 315 awake
[ 6478.161972] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105e: Two conflicting flocks from same process ========================================================== 01:19:01 \(1734398341\)
[ 6478.525380] Lustre: DEBUG MARKER: == sanity test 105e: Two conflicting flocks from same process ========================================================== 01:19:01 (1734398341)
[ 6480.392904] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105f: Enqueue same range flocks =========== 01:19:03 \(1734398343\)
[ 6480.748975] Lustre: DEBUG MARKER: == sanity test 105f: Enqueue same range flocks =========== 01:19:03 (1734398343)
[ 6483.669615] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 105g: ldlm_lock_debug stack test ========== 01:19:07 \(1734398347\)
[ 6484.033478] Lustre: DEBUG MARKER: == sanity test 105g: ldlm_lock_debug stack test ========== 01:19:07 (1734398347)
[ 6484.064007] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n ldlm.namespaces.*.dump_stack_on_error=1
[ 6484.085045] Lustre: *** cfs_fail_loc=32f, val=0***
[ 6484.086050] LustreError: 353567:0:(ldlm_flock.c:801:ldlm_flock_completion_ast()) ### Test ldlm error stack ns: lustre-MDT0000-mdc-ffff988c12f63800 lock: ffff988c133c6000/0x7bb119c3de428096 lrc: 4/0,1 mode: PW/PW res: [0x200001b78:0xd1d:0x0].0xc rrc: 2 type: FLK pid: 353566 [0->9223372036854775807] flags: 0x0 nid: local remote: 0xdf05ec28ef52388c expref: -99 pid: 353567 timeout: 0
[ 6484.091834] CPU: 1 PID: 353567 Comm: flocks_test Kdump: loaded Tainted: G           OE     5.15.0-94-generic #104-Ubuntu
[ 6484.093788] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 6484.094869] Call Trace:
[ 6484.095408]  <TASK>
[ 6484.095902]  show_stack+0x52/0x5c
[ 6484.096588]  dump_stack_lvl+0x4a/0x63
[ 6484.097325]  dump_stack+0x10/0x16
[ 6484.098009]  ldlm_flock_completion_ast.cold+0x5/0xa [ptlrpc]
[ 6484.099320]  ? unlock_res_and_lock+0x1f/0x30 [ptlrpc]
[ 6484.100346]  ? ldlm_lock_enqueue+0x20c/0x550 [ptlrpc]
[ 6484.101368]  ldlm_cli_enqueue_fini+0x9f9/0xf70 [ptlrpc]
[ 6484.102419]  ldlm_cli_enqueue+0x4df/0xb00 [ptlrpc]
[ 6484.103399]  ? ldlm_process_flock_lock+0xa50/0xa50 [ptlrpc]
[ 6484.104511]  ? mdc_changelog_cdev_finish+0x220/0x220 [mdc]
[ 6484.105600]  mdc_enqueue_base+0x691/0x2630 [mdc]
[ 6484.106514]  mdc_enqueue+0x18/0x20 [mdc]
[ 6484.107299]  lmv_enqueue+0x2a9/0x4e0 [lmv]
[ 6484.108129]  ll_file_flock+0x7d1/0x1460 [lustre]
[ 6484.109070]  ? ldlm_process_flock_lock+0xa50/0xa50 [ptlrpc]
[ 6484.110180]  ? mdc_changelog_cdev_finish+0x220/0x220 [mdc]
[ 6484.111230]  do_lock_file_wait+0x52/0x120
[ 6484.112028]  ? kmem_cache_alloc+0x1ab/0x2f0
[ 6484.112853]  ? fcntl_setlk+0x42/0x2e0
[ 6484.113595]  fcntl_setlk+0x127/0x2e0
[ 6484.114315]  do_fcntl+0x4ce/0x5a0
[ 6484.114996]  __x64_sys_fcntl+0xa9/0xd0
[ 6484.115750]  do_syscall_64+0x5c/0xc0
[ 6484.116471]  ? handle_mm_fault+0xd8/0x2c0
[ 6484.117272]  ? do_user_addr_fault+0x1e7/0x670
[ 6484.118128]  ? exit_to_user_mode_prepare+0x37/0xb0
[ 6484.119056]  ? irqentry_exit_to_user_mode+0x17/0x20
[ 6484.119996]  ? irqentry_exit+0x1d/0x30
[ 6484.120753]  ? exc_page_fault+0x89/0x170
[ 6484.121536]  entry_SYSCALL_64_after_hwframe+0x62/0xcc
[ 6484.122501] RIP: 0033:0x7f2930a6adb7
[ 6484.123233] Code: 00 00 00 89 74 24 0c 89 7c 24 08 48 89 14 24 e8 4f bc f7 ff 48 8b 14 24 8b 74 24 0c 41 89 c0 8b 7c 24 08 b8 48 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 19 44 89 c7 89 04 24 e8 96 bc f7 ff 8b 04 24
[ 6484.126496] RSP: 002b:00007f2930951d60 EFLAGS: 00000293 ORIG_RAX: 0000000000000048
[ 6484.127901] RAX: ffffffffffffffda RBX: 00007ffe10dc2610 RCX: 00007f2930a6adb7
[ 6484.129210] RDX: 00007ffe10dc2620 RSI: 0000000000000007 RDI: 0000000000000003
[ 6484.130521] RBP: 0000000000000003 R08: 0000000000000000 R09: 00007f2930951d17
[ 6484.131831] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000000007
[ 6484.133143] R13: 0000000000000000 R14: 00007f29309ea7d0 R15: 00007ffe10dc2520
[ 6484.134453]  </TASK>
[ 6487.486355] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n ldlm.namespaces.*.dump_stack_on_error=0
[ 6487.947829] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 106: attempt exec of dir followed by chown of that dir ========================================================== 01:19:11 \(1734398351\)
[ 6488.309815] Lustre: DEBUG MARKER: == sanity test 106: attempt exec of dir followed by chown of that dir ========================================================== 01:19:11 (1734398351)
[ 6490.162223] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 107: Coredump on SIG ====================== 01:19:13 \(1734398353\)
[ 6490.523215] Lustre: DEBUG MARKER: == sanity test 107: Coredump on SIG ====================== 01:19:13 (1734398353)
[ 6490.551967] Unsafe core_pattern used with fs.suid_dumpable=2.
[ 6490.551967] Pipe handler or fully qualified core dump path required.
[ 6490.551967] Set kernel.core_pattern before fs.suid_dumpable.
[ 6493.442197] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 110: filename length checking ============= 01:19:16 \(1734398356\)
[ 6493.803298] Lustre: DEBUG MARKER: == sanity test 110: filename length checking ============= 01:19:16 (1734398356)
[ 6495.710979] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 116a: stripe QOS: free space balance ============================================================================= 01:19:19 \(1734398359\)
[ 6496.074304] Lustre: DEBUG MARKER: == sanity test 116a: stripe QOS: free space balance ============================================================================= 01:19:19 (1734398359)
[ 6499.975831] Autotest: Test running for 105 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 6611.812597] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 116b: QoS shouldn\'t LBUG if not enough OSTs found on the 2nd pass ========================================================== 01:21:15 \(1734398475\)
[ 6612.262726] Lustre: DEBUG MARKER: == sanity test 116b: QoS shouldn't LBUG if not enough OSTs found on the 2nd pass ========================================================== 01:21:15 (1734398475)
[ 6615.763428] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 117: verify osd extend ==================== 01:21:19 \(1734398479\)
[ 6616.125728] Lustre: DEBUG MARKER: == sanity test 117: verify osd extend ==================== 01:21:19 (1734398479)
[ 6617.996279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118a: verify O_SYNC works ================= 01:21:21 \(1734398481\)
[ 6618.356295] Lustre: DEBUG MARKER: == sanity test 118a: verify O_SYNC works ================= 01:21:21 (1734398481)
[ 6621.018602] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118b: Reclaim dirty pages on fatal error ==================================================================== 01:21:24 \(1734398484\)
[ 6621.386678] Lustre: DEBUG MARKER: == sanity test 118b: Reclaim dirty pages on fatal error ==================================================================== 01:21:24 (1734398484)
[ 6624.712881] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_118c skipping ALWAYS excluded test 118c
[ 6625.072913] Lustre: DEBUG MARKER: SKIP: sanity test_118c skipping ALWAYS excluded test 118c
[ 6625.497350] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_118d skipping ALWAYS excluded test 118d
[ 6625.853418] Lustre: DEBUG MARKER: SKIP: sanity test_118d skipping ALWAYS excluded test 118d
[ 6626.282647] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118f: Simulate unrecoverable OSC side error ==================================================================== 01:21:29 \(1734398489\)
[ 6626.644605] Lustre: DEBUG MARKER: == sanity test 118f: Simulate unrecoverable OSC side error ==================================================================== 01:21:29 (1734398489)
[ 6627.269303] Lustre: *** cfs_fail_loc=40a, val=0***
[ 6627.270361] Lustre: Skipped 141 previous similar messages
[ 6627.271417] LustreError: 360915:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0006-osc-ffff988c12f63800: prep_req failed: rc = -22
[ 6627.273642] LustreError: 360915:0:(osc_cache.c:2129:osc_check_rpcs()) Write request failed with -22
[ 6629.287993] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118g: Don\'t stay in wait if we got local -ENOMEM  ==================================================================== 01:21:32 \(1734398492\)
[ 6629.650659] Lustre: DEBUG MARKER: == sanity test 118g: Don't stay in wait if we got local -ENOMEM ==================================================================== 01:21:32 (1734398492)
[ 6630.098856] LustreError: 361376:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0002-osc-ffff988c12f63800: prep_req failed: rc = -12
[ 6630.101197] LustreError: 361376:0:(osc_cache.c:2129:osc_check_rpcs()) Write request failed with -12
[ 6632.099079] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118h: Verify timeout in handling recoverables errors  ==================================================================== 01:21:35 \(1734398495\)
[ 6632.461363] Lustre: DEBUG MARKER: == sanity test 118h: Verify timeout in handling recoverables errors ==================================================================== 01:21:35 (1734398495)
[ 6633.654338] LustreError: lustre-OST0005-osc-ffff988c12f63800: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[ 6633.661354] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988c14bb5d40 x1818642227311872/t0(0) o4->lustre-OST0005-osc-ffff988c12f63800@10.240.22.182@tcp:6/4 lens 4584/224 e 0 to 0 dl 1734398513 ref 2 fl Interpret:RMQU/200/0 rc -5/-5 job:'multiop.0' uid:0 gid:0
[ 6633.666260] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 65 previous similar messages
[ 6634.882332] LustreError: lustre-OST0005-osc-ffff988c12f63800: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[ 6640.002358] LustreError: lustre-OST0005-osc-ffff988c12f63800: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[ 6640.004514] LustreError: Skipped 1 previous similar message
[ 6644.098614] LustreError: lustre-OST0005-osc-ffff988c12f63800: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[ 6644.100778] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) lustre-OST0005-osc-ffff988c12f63800: too many resent retries for object: 15032388560:3210: rc = -5
[ 6644.103438] LustreError: 4380:0:(osc_request.c:2602:brw_interpret()) Skipped 1 previous similar message
[ 6644.105360] Lustre: 4380:0:(llite_lib.c:4122:ll_dirty_page_discard_warn()) lustre: dirty page discard: 10.240.22.189@tcp:/lustre/fid: [0x200001b78:0x1236:0x0]// may get corrupted (rc -5)
[ 6646.503938] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118i: Fix error before timeout in recoverable error  ==================================================================== 01:21:49 \(1734398509\)
[ 6646.871822] Lustre: DEBUG MARKER: == sanity test 118i: Fix error before timeout in recoverable error ==================================================================== 01:21:49 (1734398509)
[ 6652.483299] LustreError: lustre-OST0000-osc-ffff988c12f63800: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[ 6652.485478] LustreError: Skipped 1 previous similar message
[ 6659.910394] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118j: Simulate unrecoverable OST side error ==================================================================== 01:22:03 \(1734398523\)
[ 6660.272756] Lustre: DEBUG MARKER: == sanity test 118j: Simulate unrecoverable OST side error ==================================================================== 01:22:03 (1734398523)
[ 6663.450111] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118k: bio alloc -ENOMEM and IO TERM handling =================================================================== 01:22:06 \(1734398526\)
[ 6663.812736] Lustre: DEBUG MARKER: == sanity test 118k: bio alloc -ENOMEM and IO TERM handling =================================================================== 01:22:06 (1734398526)
[ 6665.886969] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988c72892d80 x1818642227343744/t0(0) o4->lustre-OST0006-osc-ffff988c12f63800@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734398545 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'dd.0' uid:0 gid:0
[ 6665.898198] LustreError: 4380:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 15 previous similar messages
[ 6668.829676] LustreError: lustre-OST0005-osc-ffff988c12f63800: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[ 6668.831896] LustreError: Skipped 39 previous similar messages
[ 6674.932042] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118l: fsync dir =========================== 01:22:18 \(1734398538\)
[ 6675.293913] Lustre: DEBUG MARKER: == sanity test 118l: fsync dir =========================== 01:22:18 (1734398538)
[ 6677.172866] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118m: fdatasync dir ======================= 01:22:20 \(1734398540\)
[ 6677.541394] Lustre: DEBUG MARKER: == sanity test 118m: fdatasync dir ======================= 01:22:20 (1734398540)
[ 6679.415521] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 118n: statfs\(\) sends OST_STATFS requests in parallel ========================================================== 01:22:22 \(1734398542\)
[ 6679.791389] Lustre: DEBUG MARKER: == sanity test 118n: statfs() sends OST_STATFS requests in parallel ========================================================== 01:22:22 (1734398542)
[ 6684.464635] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119a: Short directIO read must return actual read amount ========================================================== 01:22:27 \(1734398547\)
[ 6684.829079] Lustre: DEBUG MARKER: == sanity test 119a: Short directIO read must return actual read amount ========================================================== 01:22:27 (1734398547)
[ 6689.364816] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119b: Sparse directIO read must return actual read amount ========================================================== 01:22:32 \(1734398552\)
[ 6689.733466] Lustre: DEBUG MARKER: == sanity test 119b: Sparse directIO read must return actual read amount ========================================================== 01:22:32 (1734398552)
[ 6692.385771] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119c: Testing for direct read hitting hole ========================================================== 01:22:35 \(1734398555\)
[ 6697.753415] Lustre: DEBUG MARKER: == sanity test 119c: Testing for direct read hitting hole ========================================================== 01:22:35 (1734398555)
[ 6699.720611] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119e: Basic tests of dio read and write at various sizes ========================================================== 01:22:43 \(1734398563\)
[ 6700.086974] Lustre: DEBUG MARKER: == sanity test 119e: Basic tests of dio read and write at various sizes ========================================================== 01:22:43 (1734398563)
[ 6717.531061] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119f: dio vs dio race ===================== 01:23:00 \(1734398580\)
[ 6717.898864] Lustre: DEBUG MARKER: == sanity test 119f: dio vs dio race ===================== 01:23:00 (1734398580)
[ 6739.519975] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119g: dio vs buffered I\/O race ============ 01:23:22 \(1734398602\)
[ 6739.892328] Lustre: DEBUG MARKER: == sanity test 119g: dio vs buffered I/O race ============ 01:23:22 (1734398602)
[ 6757.310771] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119h: basic tests of memory unaligned dio ========================================================== 01:23:40 \(1734398620\)
[ 6757.694618] Lustre: DEBUG MARKER: == sanity test 119h: basic tests of memory unaligned dio ========================================================== 01:23:40 (1734398620)
[ 6769.311411] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_119i skipping ALWAYS excluded test 119i
[ 6769.676175] Lustre: DEBUG MARKER: SKIP: sanity test_119i skipping ALWAYS excluded test 119i
[ 6770.109784] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119j: basic tests of hybrid IO switching == 01:23:53 \(1734398633\)
[ 6770.474018] Lustre: DEBUG MARKER: == sanity test 119j: basic tests of hybrid IO switching == 01:23:53 (1734398633)
[ 6771.205156] Lustre: *** cfs_fail_loc=1429, val=0***
[ 6773.131067] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119m: Test DIO readv\/writev: exercise iter duplication ========================================================== 01:23:56 \(1734398636\)
[ 6773.494942] Lustre: DEBUG MARKER: == sanity test 119m: Test DIO readv/writev: exercise iter duplication ========================================================== 01:23:56 (1734398636)
[ 6775.725068] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119n: Test Unaligned DIO readv\(\) and writev\(\) with unpatched ZFS ========================================================== 01:23:59 \(1734398639\)
[ 6776.091052] Lustre: DEBUG MARKER: == sanity test 119n: Test Unaligned DIO readv() and writev() with unpatched ZFS ========================================================== 01:23:59 (1734398639)
[ 6776.514540] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_119n need ZFS server without unaligned_dio support
[ 6776.877691] Lustre: DEBUG MARKER: SKIP: sanity test_119n need ZFS server without unaligned_dio support
[ 6777.325612] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119o: Test Unaligned DIO readv\(\) and writev\(\) with unpatched servers ========================================================== 01:24:00 \(1734398640\)
[ 6777.687624] Lustre: DEBUG MARKER: == sanity test 119o: Test Unaligned DIO readv() and writev() with unpatched servers ========================================================== 01:24:00 (1734398640)
[ 6778.105147] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_119o need ldiskfs without unaligned_dio support.
[ 6778.462395] Lustre: DEBUG MARKER: SKIP: sanity test_119o need ldiskfs without unaligned_dio support.
[ 6778.901934] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119p: Test Unaligned DIO readv\(\) and writev\(\) with patched servers ========================================================== 01:24:02 \(1734398642\)
[ 6779.263476] Lustre: DEBUG MARKER: == sanity test 119p: Test Unaligned DIO readv() and writev() with patched servers ========================================================== 01:24:02 (1734398642)
[ 6781.393097] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 119q: Test patchded Unaligned DIO readv\(\) and writev\(\) ========================================================== 01:24:04 \(1734398644\)
[ 6781.754678] Lustre: DEBUG MARKER: == sanity test 119q: Test patchded Unaligned DIO readv() and writev() ========================================================== 01:24:04 (1734398644)
[ 6787.841639] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120a: Early Lock Cancel: mkdir test ======= 01:24:11 \(1734398651\)
[ 6788.203212] Lustre: DEBUG MARKER: == sanity test 120a: Early Lock Cancel: mkdir test ======= 01:24:11 (1734398651)
[ 6790.875887] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120b: Early Lock Cancel: create test ====== 01:24:14 \(1734398654\)
[ 6791.235304] Lustre: DEBUG MARKER: == sanity test 120b: Early Lock Cancel: create test ====== 01:24:14 (1734398654)
[ 6793.803443] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120c: Early Lock Cancel: link test ======== 01:24:17 \(1734398657\)
[ 6794.168665] Lustre: DEBUG MARKER: == sanity test 120c: Early Lock Cancel: link test ======== 01:24:17 (1734398657)
[ 6796.784636] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120d: Early Lock Cancel: setattr test ===== 01:24:20 \(1734398660\)
[ 6797.148884] Lustre: DEBUG MARKER: == sanity test 120d: Early Lock Cancel: setattr test ===== 01:24:20 (1734398660)
[ 6799.738353] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120e: Early Lock Cancel: unlink test ====== 01:24:23 \(1734398663\)
[ 6800.097460] Lustre: DEBUG MARKER: == sanity test 120e: Early Lock Cancel: unlink test ====== 01:24:23 (1734398663)
[ 6800.665546] Autotest: Test running for 110 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 6809.971092] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120f: Early Lock Cancel: rename test ====== 01:24:33 \(1734398673\)
[ 6810.337991] Lustre: DEBUG MARKER: == sanity test 120f: Early Lock Cancel: rename test ====== 01:24:33 (1734398673)
[ 6820.035390] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 120g: Early Lock Cancel: performance test ========================================================== 01:24:43 \(1734398683\)
[ 6820.404872] Lustre: DEBUG MARKER: == sanity test 120g: Early Lock Cancel: performance test ========================================================== 01:24:43 (1734398683)
[ 6821.497962] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 6827.218013] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 6869.486793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 121: read cancel race ===================== 01:25:32 \(1734398732\)
[ 6869.852266] Lustre: DEBUG MARKER: == sanity test 121: read cancel race ===================== 01:25:32 (1734398732)
[ 6869.922705] Lustre: *** cfs_fail_loc=310, val=0***
[ 6871.757965] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123aa: verify statahead work ============== 01:25:35 \(1734398735\)
[ 6872.119367] Lustre: DEBUG MARKER: == sanity test 123aa: verify statahead work ============== 01:25:35 (1734398735)
[ 6873.826427] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files without statahead: 0 sec
[ 6874.186775] Lustre: DEBUG MARKER: ls -l 100 files without statahead: 0 sec
[ 6874.785830] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files with statahead: 0 sec
[ 6875.146039] Lustre: DEBUG MARKER: ls -l 100 files with statahead: 0 sec
[ 6875.823652] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 6876.979507] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 6880.462908] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files without statahead: 2 sec
[ 6880.836359] Lustre: DEBUG MARKER: ls -l 1000 files without statahead: 2 sec
[ 6882.253259] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files with statahead: 1 sec
[ 6882.617369] Lustre: DEBUG MARKER: ls -l 1000 files with statahead: 1 sec
[ 6883.294782] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 6893.521667] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 6917.885884] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files without statahead: 22 sec
[ 6918.335481] Lustre: DEBUG MARKER: ls -l 10000 files without statahead: 22 sec
[ 6928.164900] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files with statahead: 8 sec
[ 6928.528844] Lustre: DEBUG MARKER: ls -l 10000 files with statahead: 8 sec
[ 6928.947601] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l done
[ 6929.310756] Lustre: DEBUG MARKER: ls -l done
[ 6946.413279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123aa.sanity: 16 seconds
[ 6946.875593] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123aa.sanity: 16 seconds
[ 6949.055370] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123ab: verify statahead work by using statx ========================================================== 01:26:52 \(1734398812\)
[ 6949.415606] Lustre: DEBUG MARKER: == sanity test 123ab: verify statahead work by using statx ========================================================== 01:26:52 (1734398812)
[ 6951.465643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 100 files without statahead: 0 sec
[ 6951.832416] Lustre: DEBUG MARKER: statx -l 100 files without statahead: 0 sec
[ 6952.418070] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 100 files with statahead: 0 sec
[ 6952.780598] Lustre: DEBUG MARKER: statx -l 100 files with statahead: 0 sec
[ 6953.460733] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 6954.608998] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 6958.077976] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 1000 files without statahead: 2 sec
[ 6958.440536] Lustre: DEBUG MARKER: statx -l 1000 files without statahead: 2 sec
[ 6959.824864] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 1000 files with statahead: 1 sec
[ 6960.190599] Lustre: DEBUG MARKER: statx -l 1000 files with statahead: 1 sec
[ 6960.876994] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 6965.774678] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 6989.755273] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 10000 files without statahead: 22 sec
[ 6990.235224] Lustre: DEBUG MARKER: statx -l 10000 files without statahead: 22 sec
[ 6999.643293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l 10000 files with statahead: 8 sec
[ 7000.006132] Lustre: DEBUG MARKER: statx -l 10000 files with statahead: 8 sec
[ 7000.427251] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -l done
[ 7000.792290] Lustre: DEBUG MARKER: statx -l done
[ 7017.959618] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ab.sanity: 17 seconds
[ 7018.405983] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ab.sanity: 17 seconds
[ 7021.148789] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123ac: verify statahead work by using statx without glimpse RPCs ========================================================== 01:28:04 \(1734398884\)
[ 7021.511849] Lustre: DEBUG MARKER: == sanity test 123ac: verify statahead work by using statx without glimpse RPCs ========================================================== 01:28:04 (1734398884)
[ 7023.144333] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 100 files without statahead: 0 sec
[ 7023.509649] Lustre: DEBUG MARKER: statx -c %n
[ 7024.034409] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 100 files with statahead: 0 sec
[ 7024.398995] Lustre: DEBUG MARKER: statx -c %n
[ 7025.076046] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7026.240257] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7028.859216] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 1000 files without statahead: 2 sec
[ 7029.222288] Lustre: DEBUG MARKER: statx -c %n
[ 7030.247237] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 1000 files with statahead: 0 sec
[ 7030.608803] Lustre: DEBUG MARKER: statx -c %n
[ 7031.286913] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7036.554271] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7052.299040] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 10000 files without statahead: 13 sec
[ 7052.755553] Lustre: DEBUG MARKER: statx -c %n
[ 7058.772855] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 10000 files with statahead: 5 sec
[ 7059.142197] Lustre: DEBUG MARKER: statx -c %n
[ 7059.830492] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7070.634216] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7101.622845] Autotest: Test running for 115 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 7115.823053] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 30000 files without statahead: 41 sec
[ 7116.270879] Lustre: DEBUG MARKER: statx -c %n
[ 7133.450424] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D 30000 files with statahead: 13 sec
[ 7133.889769] Lustre: DEBUG MARKER: statx -c %n
[ 7134.324494] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx -c %n %i %A %h %u %g %W %X %Z -D done
[ 7134.684732] Lustre: DEBUG MARKER: statx -c %n
[ 7182.049666] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ac.sanity: 47 seconds
[ 7182.492439] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ac.sanity: 47 seconds
[ 7184.046694] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100 files without statahead: 0 sec
[ 7184.411067] Lustre: DEBUG MARKER: statx --cached=always -D 100 files without statahead: 0 sec
[ 7184.858614] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100 files with statahead: 0 sec
[ 7185.220163] Lustre: DEBUG MARKER: statx --cached=always -D 100 files with statahead: 0 sec
[ 7185.897375] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7187.047835] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7188.318393] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 1000 files without statahead: 0 sec
[ 7188.679872] Lustre: DEBUG MARKER: statx --cached=always -D 1000 files without statahead: 0 sec
[ 7189.175948] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 1000 files with statahead: 0 sec
[ 7189.536551] Lustre: DEBUG MARKER: statx --cached=always -D 1000 files with statahead: 0 sec
[ 7190.215369] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7195.366659] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7197.504904] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 10000 files without statahead: 0 sec
[ 7197.866618] Lustre: DEBUG MARKER: statx --cached=always -D 10000 files without statahead: 0 sec
[ 7198.372327] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 10000 files with statahead: 0 sec
[ 7198.731798] Lustre: DEBUG MARKER: statx --cached=always -D 10000 files with statahead: 0 sec
[ 7199.404196] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7246.739107] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7257.539099] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100000 files without statahead: 1 sec
[ 7257.905397] Lustre: DEBUG MARKER: statx --cached=always -D 100000 files without statahead: 1 sec
[ 7258.931415] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D 100000 files with statahead: 1 sec
[ 7259.295106] Lustre: DEBUG MARKER: statx --cached=always -D 100000 files with statahead: 1 sec
[ 7259.719646] Lustre: DEBUG MARKER: /usr/sbin/lctl mark statx --cached=always -D done
[ 7260.077271] Lustre: DEBUG MARKER: statx --cached=always -D done
[ 7402.815559] Autotest: Test running for 120 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 7532.933859] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ac.sanity: 273 seconds
[ 7533.377784] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ac.sanity: 273 seconds
[ 7535.930849] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:36:39 \(1734399399\)
[ 7536.298398] Lustre: DEBUG MARKER: == sanity test 123ad: Verify batching statahead works correctly ========================================================== 01:36:39 (1734399399)
[ 7538.068207] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files without statahead: 0 sec
[ 7538.429479] Lustre: DEBUG MARKER: ls -l 100 files without statahead: 0 sec
[ 7539.035376] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files with statahead: 0 sec
[ 7539.400295] Lustre: DEBUG MARKER: ls -l 100 files with statahead: 0 sec
[ 7540.085306] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7541.235379] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7544.786097] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files without statahead: 3 sec
[ 7545.148937] Lustre: DEBUG MARKER: ls -l 1000 files without statahead: 3 sec
[ 7547.017404] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files with statahead: 1 sec
[ 7547.377565] Lustre: DEBUG MARKER: ls -l 1000 files with statahead: 1 sec
[ 7548.066535] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7553.562179] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7578.244274] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files without statahead: 22 sec
[ 7578.687361] Lustre: DEBUG MARKER: ls -l 10000 files without statahead: 22 sec
[ 7593.369193] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files with statahead: 12 sec
[ 7593.816680] Lustre: DEBUG MARKER: ls -l 10000 files with statahead: 12 sec
[ 7594.240438] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l done
[ 7594.600734] Lustre: DEBUG MARKER: ls -l done
[ 7612.023114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ad.sanity: 17 seconds
[ 7612.472592] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ad.sanity: 17 seconds
[ 7613.161876] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7618.878167] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7631.006561] Lustre: DEBUG MARKER: pgrep ll_sa
[ 7645.288236] Lustre: DEBUG MARKER: pgrep ll_sa
[ 7674.670917] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files without statahead: 1 sec
[ 7675.114039] Lustre: DEBUG MARKER: ls -l 100 files without statahead: 1 sec
[ 7675.705151] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 100 files with statahead: 0 sec
[ 7676.069502] Lustre: DEBUG MARKER: ls -l 100 files with statahead: 0 sec
[ 7676.751846] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7677.902267] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7681.471030] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files without statahead: 2 sec
[ 7681.829557] Lustre: DEBUG MARKER: ls -l 1000 files without statahead: 2 sec
[ 7683.425060] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 1000 files with statahead: 1 sec
[ 7683.787605] Lustre: DEBUG MARKER: ls -l 1000 files with statahead: 1 sec
[ 7684.471462] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7689.738364] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7703.758739] Autotest: Test running for 125 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 7714.416063] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files without statahead: 22 sec
[ 7714.860218] Lustre: DEBUG MARKER: ls -l 10000 files without statahead: 22 sec
[ 7724.298836] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l 10000 files with statahead: 7 sec
[ 7724.664740] Lustre: DEBUG MARKER: ls -l 10000 files with statahead: 7 sec
[ 7725.091200] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -l done
[ 7725.455723] Lustre: DEBUG MARKER: ls -l done
[ 7742.862064] Lustre: DEBUG MARKER: /usr/sbin/lctl mark rm -r \/mnt\/lustre\/d123ad.sanity: 17 seconds
[ 7743.322429] Lustre: DEBUG MARKER: rm -r /mnt/lustre/d123ad.sanity: 17 seconds
[ 7744.027198] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7750.164641] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7761.596589] Lustre: DEBUG MARKER: pgrep ll_sa
[ 7770.887360] Lustre: DEBUG MARKER: pgrep ll_sa
[ 7790.064959] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123b: not panic with network error in statahead enqueue \(bug 15027\) ========================================================== 01:40:53 \(1734399653\)
[ 7790.518071] Lustre: DEBUG MARKER: == sanity test 123b: not panic with network error in statahead enqueue (bug 15027) ========================================================== 01:40:53 (1734399653)
[ 7796.203919] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 7797.449178] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 7799.585504] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls done
[ 7799.944906] Lustre: DEBUG MARKER: ls done
[ 7804.103258] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123c: Can not initialize inode warning on DNE statahead ========================================================== 01:41:07 \(1734399667\)
[ 7804.466121] Lustre: DEBUG MARKER: == sanity test 123c: Can not initialize inode warning on DNE statahead ========================================================== 01:41:07 (1734399667)
[ 7804.886355] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_123c needs \>= 2 MDTs
[ 7805.250468] Lustre: DEBUG MARKER: SKIP: sanity test_123c needs >= 2 MDTs
[ 7805.697490] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123d: Statahead on striped directories works correctly ========================================================== 01:41:09 \(1734399669\)
[ 7806.062178] Lustre: DEBUG MARKER: == sanity test 123d: Statahead on striped directories works correctly ========================================================== 01:41:09 (1734399669)
[ 7806.287518] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 7806.300085] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 7806.477435] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 7806.627222] Lustre: Unmounted lustre-client
[ 7806.637643] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 7806.650128] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 7806.702227] Lustre: Mounted lustre-client
[ 7809.783897] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123e: statahead with large wide striping == 01:41:13 \(1734399673\)
[ 7810.148919] Lustre: DEBUG MARKER: == sanity test 123e: statahead with large wide striping == 01:41:13 (1734399673)
[ 7839.342273] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123f: Retry mechanism with large wide striping files ========================================================== 01:41:42 \(1734399702\)
[ 7840.300809] Lustre: DEBUG MARKER: == sanity test 123f: Retry mechanism with large wide striping files ========================================================== 01:41:42 (1734399702)
[ 8003.497957] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123g: Test for stat-ahead advise ========== 01:44:26 \(1734399866\)
[ 8004.341160] Autotest: Test running for 130 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8005.843404] Lustre: DEBUG MARKER: == sanity test 123g: Test for stat-ahead advise ========== 01:44:26 (1734399866)
[ 8007.123867] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8010.583423] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8015.132028] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8017.373025] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123h: Verify statahead work with the fname pattern via du ========================================================== 01:44:40 \(1734399880\)
[ 8017.861650] Lustre: DEBUG MARKER: == sanity test 123h: Verify statahead work with the fname pattern via du ========================================================== 01:44:40 (1734399880)
[ 8093.950967] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8094.977682] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8179.159286] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8180.186945] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8198.612023] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123i: Verify statahead work with the fname indexing pattern ========================================================== 01:47:42 \(1734400062\)
[ 8198.977079] Lustre: DEBUG MARKER: == sanity test 123i: Verify statahead work with the fname indexing pattern ========================================================== 01:47:42 (1734400062)
[ 8199.703811] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8200.809739] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8202.106558] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8203.130934] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8204.153841] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8205.176654] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8206.199236] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8207.222053] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8208.244807] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8209.267328] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8210.289507] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8211.312107] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8212.334723] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8213.357380] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8214.380132] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8215.402896] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8216.425004] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8217.446072] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8218.467496] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8219.489936] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8220.512259] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8221.534459] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8222.557107] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8223.579730] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8224.602282] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8225.624983] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8226.647121] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8227.669349] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8228.691810] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8229.714851] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8230.737977] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8231.761259] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8232.783819] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8233.806107] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8234.829118] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8235.587893] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8236.625497] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8237.923319] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8238.950250] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8239.973338] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8240.996218] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8242.019155] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8243.041901] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8244.064872] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8245.087583] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8246.109592] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8247.131479] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8248.154475] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8249.177336] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8250.200006] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8251.222953] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8252.246101] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8253.269096] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8254.291292] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8255.313333] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8256.336092] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8257.358893] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8258.381742] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8259.404561] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8260.427572] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8261.450409] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8262.473300] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8263.496044] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8264.518684] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8265.540601] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8266.562370] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8267.585113] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8268.607868] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8269.630614] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8271.800565] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8272.735844] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8274.169737] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8275.278154] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8276.474587] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123j: -ENOENT error from batched statahead be handled correctly ========================================================== 01:48:59 \(1734400139\)
[ 8276.843508] Lustre: DEBUG MARKER: == sanity test 123j: -ENOENT error from batched statahead be handled correctly ========================================================== 01:48:59 (1734400139)
[ 8277.252999] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_123j needs \>= 2 MDTs
[ 8277.611596] Lustre: DEBUG MARKER: SKIP: sanity test_123j needs >= 2 MDTs
[ 8278.051623] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123k: Verify statahead work with mdtest shared stat\(\) mode ========================================================== 01:49:01 \(1734400141\)
[ 8278.412836] Lustre: DEBUG MARKER: == sanity test 123k: Verify statahead work with mdtest shared stat() mode ========================================================== 01:49:01 (1734400141)
[ 8287.114992] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8288.139346] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8289.162244] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8290.185257] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8291.208190] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8292.231200] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8293.254120] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8294.277106] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8295.300195] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8296.323369] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8297.346586] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8298.369793] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8299.392028] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8300.413638] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8301.436814] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8302.461163] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8303.485488] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8304.509551] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8305.532431] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8306.430871] Autotest: Test running for 135 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8306.554681] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8307.577656] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8308.600721] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8309.623350] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8310.645743] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8311.667206] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8312.687661] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8313.707920] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8314.728373] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8315.748732] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8316.769832] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8317.792016] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8318.812822] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8320.757666] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 123l: Avoid panic when revalidate a local cached entry ========================================================== 01:49:44 \(1734400184\)
[ 8321.122931] Lustre: DEBUG MARKER: == sanity test 123l: Avoid panic when revalidate a local cached entry ========================================================== 01:49:44 (1734400184)
[ 8321.941306] LustreError: 396567:0:(statahead.c:371:sa_put()) cfs_fail_timeout id 1433 sleeping for 35000ms
[ 8356.990669] LustreError: 396567:0:(statahead.c:371:sa_put()) cfs_fail_timeout id 1433 awake
[ 8357.148617] Lustre: DEBUG MARKER: pgrep ll_sa
[ 8359.248932] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124a: lru resize ================================================================================================= 01:50:22 \(1734400222\)
[ 8359.615936] Lustre: DEBUG MARKER: == sanity test 124a: lru resize ================================================================================================= 01:50:22 (1734400222)
[ 8360.047607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark create 2000 files at \/mnt\/lustre\/d124a.sanity
[ 8360.408702] Lustre: DEBUG MARKER: create 2000 files at /mnt/lustre/d124a.sanity
[ 8361.077850] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8363.580089] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8366.592984] Lustre: DEBUG MARKER: /usr/sbin/lctl mark NSDIR=ldlm.namespaces.lustre-MDT0000-mdc-ffff988c01424800
[ 8366.959417] Lustre: DEBUG MARKER: NSDIR=ldlm.namespaces.lustre-MDT0000-mdc-ffff988c01424800
[ 8367.376754] Lustre: DEBUG MARKER: /usr/sbin/lctl mark NS=ldlm.namespaces.lustre-MDT0000-mdc-ffff988c01424800
[ 8367.736200] Lustre: DEBUG MARKER: NS=ldlm.namespaces.lustre-MDT0000-mdc-ffff988c01424800
[ 8368.154346] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LRU=2003
[ 8368.513870] Lustre: DEBUG MARKER: LRU=2003
[ 8368.931813] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LIMIT=180049
[ 8369.291383] Lustre: DEBUG MARKER: LIMIT=180049
[ 8369.703529] Lustre: DEBUG MARKER: /usr/sbin/lctl mark LVF=10786700
[ 8370.062548] Lustre: DEBUG MARKER: LVF=10786700
[ 8370.478133] Lustre: DEBUG MARKER: /usr/sbin/lctl mark OLD_LVF=100
[ 8370.839594] Lustre: DEBUG MARKER: OLD_LVF=100
[ 8371.252143] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Sleep 50 sec
[ 8371.613080] Lustre: DEBUG MARKER: Sleep 50 sec
[ 8422.108011] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Dropped 825 locks in 50s
[ 8422.553094] Lustre: DEBUG MARKER: Dropped 825 locks in 50s
[ 8422.978601] Lustre: DEBUG MARKER: /usr/sbin/lctl mark unlink 2000 files at \/mnt\/lustre\/d124a.sanity
[ 8423.344469] Lustre: DEBUG MARKER: unlink 2000 files at /mnt/lustre/d124a.sanity
[ 8424.026482] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8425.423426] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8428.038926] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124b: lru resize \(performance test\) ================================================================================= 01:51:31 \(1734400291\)
[ 8428.415729] Lustre: DEBUG MARKER: == sanity test 124b: lru resize (performance test) ================================================================================= 01:51:31 (1734400291)
[ 8429.158190] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8438.553612] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8439.710275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark doing ls -la \/mnt\/lustre\/d124b.sanity\/disable_lru_resize 3 times
[ 8440.072754] Lustre: DEBUG MARKER: doing ls -la /mnt/lustre/d124b.sanity/disable_lru_resize 3 times
[ 8451.295224] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -la time: 11 seconds
[ 8451.756311] Lustre: DEBUG MARKER: ls -la time: 11 seconds
[ 8452.177115] Lustre: DEBUG MARKER: /usr/sbin/lctl mark lru_size = 200
[ 8452.546133] Lustre: DEBUG MARKER: lru_size = 200
[ 8453.223404] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8455.598678] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8457.398936] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8461.739244] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8462.912534] Lustre: DEBUG MARKER: /usr/sbin/lctl mark doing ls -la \/mnt\/lustre\/d124b.sanity\/enable_lru_resize 3 times
[ 8463.272206] Lustre: DEBUG MARKER: doing ls -la /mnt/lustre/d124b.sanity/enable_lru_resize 3 times
[ 8469.850739] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -la time: 6 seconds
[ 8470.216444] Lustre: DEBUG MARKER: ls -la time: 6 seconds
[ 8470.639035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark lru_size = 4006
[ 8471.004630] Lustre: DEBUG MARKER: lru_size = 4006
[ 8471.422694] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls -la is 45% faster with lru resize enabled
[ 8471.786212] Lustre: DEBUG MARKER: ls -la is 45% faster with lru resize enabled
[ 8473.873483] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8475.599335] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8476.809255] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124c: LRUR cancel very aged locks ========= 01:52:20 \(1734400340\)
[ 8477.171537] Lustre: DEBUG MARKER: == sanity test 124c: LRUR cancel very aged locks ========= 01:52:20 (1734400340)
[ 8500.140029] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 124d: cancel very aged locks if lru-resize disabled ========================================================== 01:52:43 \(1734400363\)
[ 8500.511857] Lustre: DEBUG MARKER: == sanity test 124d: cancel very aged locks if lru-resize disabled ========================================================== 01:52:43 (1734400363)
[ 8523.180252] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 125: don\'t return EPROTO when a dir has a non-default striping and ACLs ========================================================== 01:53:06 \(1734400386\)
[ 8523.546720] Lustre: DEBUG MARKER: == sanity test 125: don't return EPROTO when a dir has a non-default striping and ACLs ========================================================== 01:53:06 (1734400386)
[ 8525.421948] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 126: check that the fsgid provided by the client is taken into account ========================================================== 01:53:08 \(1734400388\)
[ 8525.787045] Lustre: DEBUG MARKER: == sanity test 126: check that the fsgid provided by the client is taken into account ========================================================== 01:53:08 (1734400388)
[ 8527.641128] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 127a: verify the client stats are sane ==== 01:53:11 \(1734400391\)
[ 8527.999212] Lustre: DEBUG MARKER: == sanity test 127a: verify the client stats are sane ==== 01:53:11 (1734400391)
[ 8530.187141] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 127b: verify the llite client stats are sane ========================================================== 01:53:13 \(1734400393\)
[ 8530.551013] Lustre: DEBUG MARKER: == sanity test 127b: verify the llite client stats are sane ========================================================== 01:53:13 (1734400393)
[ 8532.483531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 127c: test llite extent stats with regular & mmap i\/o ========================================================== 01:53:15 \(1734400395\)
[ 8532.848213] Lustre: DEBUG MARKER: == sanity test 127c: test llite extent stats with regular
[ 8542.662643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 128: interactive lfs for 2 consecutive find\'s ========================================================== 01:53:26 \(1734400406\)
[ 8543.025350] Lustre: DEBUG MARKER: == sanity test 128: interactive lfs for 2 consecutive find's ========================================================== 01:53:26 (1734400406)
[ 8544.873389] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 129: test directory size limit ================================================================================== 01:53:28 \(1734400408\)
[ 8545.234135] Lustre: DEBUG MARKER: == sanity test 129: test directory size limit ================================================================================== 01:53:28 (1734400408)
[ 8553.941381] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8554.823414] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8557.242024] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130a: FIEMAP \(1-stripe file\) ============== 01:53:40 \(1734400420\)
[ 8557.603750] Lustre: DEBUG MARKER: == sanity test 130a: FIEMAP (1-stripe file) ============== 01:53:40 (1734400420)
[ 8559.515922] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130b: FIEMAP \(7-stripe file\) ============== 01:53:42 \(1734400422\)
[ 8559.876736] Lustre: DEBUG MARKER: == sanity test 130b: FIEMAP (7-stripe file) ============== 01:53:42 (1734400422)
[ 8562.137565] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130c: FIEMAP \(2-stripe file with hole\) ==== 01:53:45 \(1734400425\)
[ 8562.502650] Lustre: DEBUG MARKER: == sanity test 130c: FIEMAP (2-stripe file with hole) ==== 01:53:45 (1734400425)
[ 8564.445377] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130d: FIEMAP \(N-stripe file\) ============== 01:53:47 \(1734400427\)
[ 8564.804331] Lustre: DEBUG MARKER: == sanity test 130d: FIEMAP (N-stripe file) ============== 01:53:47 (1734400427)
[ 8567.334371] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130e: FIEMAP \(test continuation FIEMAP calls\) ========================================================== 01:53:50 \(1734400430\)
[ 8567.693481] Lustre: DEBUG MARKER: == sanity test 130e: FIEMAP (test continuation FIEMAP calls) ========================================================== 01:53:50 (1734400430)
[ 8578.820304] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130f: FIEMAP \(unstriped file\) ============= 01:54:02 \(1734400442\)
[ 8579.184865] Lustre: DEBUG MARKER: == sanity test 130f: FIEMAP (unstriped file) ============= 01:54:02 (1734400442)
[ 8581.063335] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130g: FIEMAP \(overstripe file\) ============ 01:54:04 \(1734400444\)
[ 8581.427662] Lustre: DEBUG MARKER: == sanity test 130g: FIEMAP (overstripe file) ============ 01:54:04 (1734400444)
[ 8607.548212] Autotest: Test running for 140 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8611.784758] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130h: FIEMAP deadlock ===================== 01:54:35 \(1734400475\)
[ 8612.143136] Lustre: DEBUG MARKER: == sanity test 130h: FIEMAP deadlock ===================== 01:54:35 (1734400475)
[ 8613.811883] LustreError: 413831:0:(osc_object.c:274:osc_object_fiemap()) cfs_fail_timeout id 418 sleeping for 5000ms
[ 8618.909158] LustreError: 413831:0:(osc_object.c:274:osc_object_fiemap()) cfs_fail_timeout id 418 awake
[ 8620.725813] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 130i: FIEMAP \(DoM file\) =================== 01:54:44 \(1734400484\)
[ 8621.087494] Lustre: DEBUG MARKER: == sanity test 130i: FIEMAP (DoM file) =================== 01:54:44 (1734400484)
[ 8628.522649] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131a: test iov\'s crossing stripe boundary for writev\/readv ========================================================== 01:54:51 \(1734400491\)
[ 8628.887895] Lustre: DEBUG MARKER: == sanity test 131a: test iov's crossing stripe boundary for writev/readv ========================================================== 01:54:51 (1734400491)
[ 8630.799175] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131b: test append writev ================== 01:54:54 \(1734400494\)
[ 8631.161376] Lustre: DEBUG MARKER: == sanity test 131b: test append writev ================== 01:54:54 (1734400494)
[ 8633.173766] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131c: test read\/write on file w\/o objects ========================================================== 01:54:56 \(1734400496\)
[ 8633.537354] Lustre: DEBUG MARKER: == sanity test 131c: test read/write on file w/o objects ========================================================== 01:54:56 (1734400496)
[ 8635.381892] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131d: test short read ===================== 01:54:58 \(1734400498\)
[ 8635.743069] Lustre: DEBUG MARKER: == sanity test 131d: test short read ===================== 01:54:58 (1734400498)
[ 8637.634054] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 131e: test read hitting hole ============== 01:55:01 \(1734400501\)
[ 8637.996191] Lustre: DEBUG MARKER: == sanity test 131e: test read hitting hole ============== 01:55:01 (1734400501)
[ 8639.923296] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133a: Verifying MDT stats ================================================================================================== 01:55:03 \(1734400503\)
[ 8640.292704] Lustre: DEBUG MARKER: == sanity test 133a: Verifying MDT stats ================================================================================================== 01:55:03 (1734400503)
[ 8647.010303] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133b: Verifying extra MDT stats ============================================================================================ 01:55:10 \(1734400510\)
[ 8647.368675] Lustre: DEBUG MARKER: == sanity test 133b: Verifying extra MDT stats ============================================================================================ 01:55:10 (1734400510)
[ 8657.169319] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133c: Verifying OST stats ================================================================================================== 01:55:20 \(1734400520\)
[ 8657.537444] Lustre: DEBUG MARKER: == sanity test 133c: Verifying OST stats ================================================================================================== 01:55:20 (1734400520)
[ 8670.914315] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133d: Verifying rename_stats ================================================================================================== 01:55:34 \(1734400534\)
[ 8671.286937] Lustre: DEBUG MARKER: == sanity test 133d: Verifying rename_stats ================================================================================================== 01:55:34 (1734400534)
[ 8672.674337] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 8673.738280] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 8679.619452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133e: Verifying OST {read,write}_bytes nid stats =========================================================================== 01:55:43 \(1734400543\)
[ 8679.985778] Lustre: DEBUG MARKER: == sanity test 133e: Verifying OST read_bytes write_bytes nid stats =========================================================================== 01:55:43 (1734400543)
[ 8684.129064] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133f: Check reads\/writes of client lustre proc files with bad area io ========================================================== 01:55:47 \(1734400547\)
[ 8684.494461] Lustre: DEBUG MARKER: == sanity test 133f: Check reads/writes of client lustre proc files with bad area io ========================================================== 01:55:47 (1734400547)
[ 8684.509407] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[ 8687.610824] Lustre: 424550:0:(libcfs_string.c:161:cfs_str2mask()) unknown mask ''.
[ 8687.610824] mask usage: [+|-]<all|type> ...
[ 8687.618251] ------------[ cut here ]------------
[ 8687.619170] WARNING: CPU: 1 PID: 424550 at mm/page_alloc.c:5396 __alloc_pages+0x2b0/0x330
[ 8687.620698] Modules linked in: tcp_diag inet_diag lustre(OE) obdecho(OE) mgc(OE) mdc(OE) lov(OE) osc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 nfsv4 nfs fscache netfs intel_rapl_msr intel_rapl_common joydev input_leds mac_hid serio_raw qemu_fw_cfg binfmt_misc sch_fq_codel dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua msr efi_pstore drm nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel virtio_net aesni_intel crypto_simd net_failover cryptd psmouse virtio_blk failover i2c_piix4 pata_acpi floppy [last unloaded: obd_test]
[ 8687.632999] CPU: 1 PID: 424550 Comm: badarea_io Kdump: loaded Tainted: G           OE     5.15.0-94-generic #104-Ubuntu
[ 8687.634910] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 8687.635990] RIP: 0010:__alloc_pages+0x2b0/0x330
[ 8687.636888] Code: 48 8b 04 25 c0 fb 01 00 48 05 a0 0d 00 00 41 be 01 00 00 00 48 89 45 b0 e9 45 fe ff ff 41 81 e4 00 20 00 00 0f 85 60 ff ff ff <0f> 0b e9 59 ff ff ff 31 c0 e9 19 fe ff ff f7 c2 00 00 08 00 75 53
[ 8687.640135] RSP: 0018:ffffba88470cbbf8 EFLAGS: 00010246
[ 8687.641134] RAX: 0000000000000000 RBX: ffffffff9b93cda0 RCX: 0000000000000000
[ 8687.642434] RDX: 0000000000000000 RSI: 0000000000000013 RDI: 0000000000040cc0
[ 8687.643745] RBP: ffffba88470cbc50 R08: ffffba88470cbe58 R09: 0000000000000000
[ 8687.645051] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 8687.646354] R13: 0000000000000013 R14: 0000000000000000 R15: 0000000000000000
[ 8687.647658] FS:  00007fcfecd1b740(0000) GS:ffff988d3bd00000(0000) knlGS:0000000000000000
[ 8687.649132] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8687.650220] CR2: 00007ffe294cd000 CR3: 00000000145ec002 CR4: 00000000003706e0
[ 8687.651521] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8687.652842] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 8687.654240] Call Trace:
[ 8687.654793]  <TASK>
[ 8687.655279]  ? show_trace_log_lvl+0x1d6/0x2ea
[ 8687.656140]  ? show_trace_log_lvl+0x1d6/0x2ea
[ 8687.657007]  ? alloc_pages+0x9e/0x1e0
[ 8687.657746]  ? show_regs.part.0+0x23/0x29
[ 8687.658534]  ? show_regs.cold+0x8/0xd
[ 8687.659270]  ? __alloc_pages+0x2b0/0x330
[ 8687.660053]  ? __warn+0x8c/0x100
[ 8687.660725]  ? __alloc_pages+0x2b0/0x330
[ 8687.661521]  ? report_bug+0xa4/0xd0
[ 8687.662236]  ? handle_bug+0x39/0x90
[ 8687.662940]  ? exc_invalid_op+0x19/0x70
[ 8687.663705]  ? asm_exc_invalid_op+0x1b/0x20
[ 8687.664531]  ? __alloc_pages+0x2b0/0x330
[ 8687.665346]  alloc_pages+0x9e/0x1e0
[ 8687.666126]  kmalloc_order+0x2f/0xd0
[ 8687.666902]  kmalloc_order_trace+0x1d/0x90
[ 8687.667718]  __kmalloc_track_caller+0x2c1/0x340
[ 8687.668594]  ? proc_dobitmasks+0xa9/0x110 [libcfs]
[ 8687.669580]  memdup_user_nul+0x26/0xa0
[ 8687.670336]  proc_dobitmasks+0xa9/0x110 [libcfs]
[ 8687.671232]  lnet_debugfs_write+0x3e/0x90 [libcfs]
[ 8687.672155]  full_proxy_write+0x5c/0x90
[ 8687.672929]  vfs_write+0xc7/0x270
[ 8687.673614]  ksys_write+0x67/0xf0
[ 8687.674368]  __x64_sys_write+0x19/0x20
[ 8687.675126]  do_syscall_64+0x5c/0xc0
[ 8687.675846]  ? syscall_exit_to_user_mode+0x35/0x50
[ 8687.676774]  ? __x64_sys_write+0x19/0x20
[ 8687.677551]  ? do_syscall_64+0x69/0xc0
[ 8687.678306]  ? exit_to_user_mode_prepare+0x37/0xb0
[ 8687.679226]  ? syscall_exit_to_user_mode+0x35/0x50
[ 8687.680146]  ? __x64_sys_write+0x19/0x20
[ 8687.680932]  ? do_syscall_64+0x69/0xc0
[ 8687.681683]  ? exc_page_fault+0x89/0x170
[ 8687.682458]  entry_SYSCALL_64_after_hwframe+0x62/0xcc
[ 8687.683423] RIP: 0033:0x7fcfece32887
[ 8687.684163] Code: 10 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 8687.687433] RSP: 002b:00007ffe294c8308 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[ 8687.688844] RAX: ffffffffffffffda RBX: 00007ffe294c8448 RCX: 00007fcfece32887
[ 8687.690155] RDX: 0000000060000000 RSI: 00007ffe294c8314 RDI: 0000000000000003
[ 8687.691457] RBP: 0000000000000002 R08: 00007fcfecf39f10 R09: 00007fcfecf57040
[ 8687.692766] R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffe294c8448
[ 8687.694069] R13: 000055b84e6530e0 R14: 000055b84e655da0 R15: 00007fcfecf8b040
[ 8687.695367]  </TASK>
[ 8687.695864] ---[ end trace 2a92d4601bfcfaf3 ]---
[ 8687.699140] Lustre: DEBUG MARKER: 
[ 8687.699876] Lustre: DEBUG MARKER: 
[ 8707.701444] Lustre: lustre-MDT0000-mdc-ffff988c01424800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 8707.708997] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[ 8707.712725] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f0a30dd4 to 0xdf05ec28f0e05f2a
[ 8707.715253] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[ 8707.717040] Lustre: Skipped 1 previous similar message
[ 8708.724659] Lustre: 4381:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734400556/real 1734400556]  req@ffff988c14a60340 x1818642562679424/t0(0) o400->lustre-MDT0000-mdc-ffff988c01424800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734400572 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[ 8708.955939] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[ 8710.213102] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[ 8710.666651] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[ 8710.742314] Lustre: DEBUG MARKER: lctl get_param -n at_max
[ 8711.140268] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[ 8711.584953] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[ 8713.687729] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 8713.687729] if [ $running -ne 0 ] ; then
[ 8713.687729] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 8713.687729] lsof /mnt/lustre || need_kill=no;
[ 8713.687729] if [ x != x -a x$need_kill != xno ]; then
[ 8713.687729]     pids=$(lsof -t /mnt/lustre | sort -u);
[ 8713.687729]     if 
[ 8713.844630] Lustre: 4380:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734400561/real 1734400561]  req@ffff988c14a623c0 x1818642562680576/t0(0) o400->lustre-MDT0000-mdc-ffff988c01424800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734400577 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[ 8715.236459] Lustre: Unmounted lustre-client
[ 8715.568934] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 8715.568934] if [ $running -ne 0 ] ; then
[ 8715.568934] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 8715.568934] lsof /mnt/lustre2 || need_kill=no;
[ 8715.568934] if [ x != x -a x$need_kill != xno ]; then
[ 8715.568934]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 8715.568934]    
[ 8746.661468] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[ 8746.661468] lctl dl | grep ' ST ' || true
[ 8747.076969] Key type lgssc unregistered
[ 8747.156564] LNet: 426627:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 8748.196526] LNet: Removed LNI 10.240.22.180@tcp
[ 8748.428851] Key type .llcrypt unregistered
[ 8748.429752] Key type ._llcrypt unregistered
[ 8748.917195] Key type ._llcrypt registered
[ 8748.918128] Key type .llcrypt registered
[ 8748.978591] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[ 8749.792306] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 8749.836227] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[ 8749.837492] LNet: Accept all, port 7988
[ 8751.376392] Key type lgssc registered
[ 8751.554181] Lustre: Echo OBD driver; http://www.lustre.org/
[ 8755.257674] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 8755.257674] mpts=$(mount | grep -c /mnt/lustre' ');
[ 8755.257674] if [ $running -ne $mpts ]; then
[ 8755.257674]     echo $(hostname) env are INSANE!;
[ 8755.257674]     exit 1;
[ 8755.257674] fi
[ 8756.038555] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 8756.038555] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 8756.038555] if [ $running -ne $mpts ]; then
[ 8756.038555]     echo $(hostname) env are INSANE!;
[ 8756.038555]     exit 1;
[ 8756.038555] fi
[ 8813.648407] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 8813.663202] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 8814.705478] Lustre: Mounted lustre-client
[ 8815.049579] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 8815.734290] Lustre: DEBUG MARKER: 
[ 8815.734290] running=$(mount | grep -c /mnt/lustre' ');
[ 8815.734290] rc=0;
[ 8815.734290] if [ $running -eq 0 ] ; then
[ 8815.734290] 	mkdir -p /mnt/lustre;
[ 8815.734290] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[ 8815.734290] 	rc=$?;
[ 8815.734290] fi;
[ 8815.734290] exit $rc
[ 8816.497331] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 8821.220240] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 8821.581309] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 8821.994857] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 8822.688212] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 8823.058384] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[ 8823.071018] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[ 8832.443347] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133g: Check reads\/writes of server lustre proc files with bad area io ========================================================== 01:58:15 \(1734400695\)
[ 8832.810518] Lustre: DEBUG MARKER: == sanity test 133g: Check reads/writes of server lustre proc files with bad area io ========================================================== 01:58:15 (1734400695)
[ 8840.307978] Lustre: lustre-OST0000-osc-ffff988c67e2b800: disconnect after 20s idle
[ 8844.301057] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 8844.301057] if [ $running -ne 0 ] ; then
[ 8844.301057] echo Stopping client $(hostname) /mnt/lustre opts:;
[ 8844.301057] lsof /mnt/lustre || need_kill=no;
[ 8844.301057] if [ x != x -a x$need_kill != xno ]; then
[ 8844.301057]     pids=$(lsof -t /mnt/lustre | sort -u);
[ 8844.301057]     if 
[ 8845.529460] Lustre: Unmounted lustre-client
[ 8845.861073] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 8845.861073] if [ $running -ne 0 ] ; then
[ 8845.861073] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[ 8845.861073] lsof /mnt/lustre2 || need_kill=no;
[ 8845.861073] if [ x != x -a x$need_kill != xno ]; then
[ 8845.861073]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[ 8845.861073]    
[ 8875.704886] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1 || true
[ 8877.105194] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[ 8877.105194] lctl dl | grep ' ST ' || true
[ 8877.507518] Key type lgssc unregistered
[ 8877.583786] LNet: 432141:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[ 8878.627811] LNet: Removed LNI 10.240.22.180@tcp
[ 8878.847958] Key type .llcrypt unregistered
[ 8878.848875] Key type ._llcrypt unregistered
[ 8879.321506] Key type ._llcrypt registered
[ 8879.322427] Key type .llcrypt registered
[ 8879.373735] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[ 8880.184497] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[ 8880.228672] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[ 8880.229897] LNet: Accept all, port 7988
[ 8881.763636] Key type lgssc registered
[ 8881.930291] Lustre: Echo OBD driver; http://www.lustre.org/
[ 8885.582112] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[ 8885.582112] mpts=$(mount | grep -c /mnt/lustre' ');
[ 8885.582112] if [ $running -ne $mpts ]; then
[ 8885.582112]     echo $(hostname) env are INSANE!;
[ 8885.582112]     exit 1;
[ 8885.582112] fi
[ 8886.281267] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[ 8886.281267] mpts=$(mount | grep -c /mnt/lustre2' ');
[ 8886.281267] if [ $running -ne $mpts ]; then
[ 8886.281267]     echo $(hostname) env are INSANE!;
[ 8886.281267]     exit 1;
[ 8886.281267] fi
[ 8908.359134] Autotest: Test running for 145 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 8943.400789] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 8943.416837] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 8944.468826] Lustre: Mounted lustre-client
[ 8944.815626] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 8945.499123] Lustre: DEBUG MARKER: 
[ 8945.499123] running=$(mount | grep -c /mnt/lustre' ');
[ 8945.499123] rc=0;
[ 8945.499123] if [ $running -eq 0 ] ; then
[ 8945.499123] 	mkdir -p /mnt/lustre;
[ 8945.499123] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[ 8945.499123] 	rc=$?;
[ 8945.499123] fi;
[ 8945.499123] exit $rc
[ 8946.250554] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[ 8950.982980] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[ 8951.350895] Lustre: DEBUG MARKER: Using TIMEOUT=20
[ 8951.765322] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[ 8952.454669] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[ 8959.656404] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 133h: Proc files should end with newlines ========================================================== 02:00:23 \(1734400823\)
[ 8960.022992] Lustre: DEBUG MARKER: == sanity test 133h: Proc files should end with newlines ========================================================== 02:00:23 (1734400823)
[ 8960.126165] Lustre: DEBUG MARKER: 
[ 8960.159437] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_early_margin | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.177147] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_extra | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.194754] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.212333] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.229961] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.247666] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.265390] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_peer_on_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.283300] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_raw_pointers | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.301014] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param dump_on_eviction | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.318704] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param dump_on_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.336434] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param enable_stats_header | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.354180] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param evict_multiplier | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.371863] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param jobid_name | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.389425] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param jobid_this_session | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.407086] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param jobid_var | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.424637] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lbug_on_eviction | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.442397] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.cancel_unused_locks_before_replay | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.460396] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.478637] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.496919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.515192] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.533398] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.551565] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.569628] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.587838] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.606405] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.624662] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.642919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.MGC10.240.22.189@tcp.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.660964] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.679431] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.698250] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.716722] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.735263] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.753591] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.776480] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.795380] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.813959] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.832664] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.851329] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-MDT0000-mdc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.869832] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.888335] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.906917] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.925438] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.944118] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.962589] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.981057] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8960.999584] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.018096] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.036714] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.055462] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0000-osc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.074003] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.092562] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.111237] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.129800] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.148365] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.166927] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.185461] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.204002] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.222586] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.241232] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.259874] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0001-osc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.278398] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.297008] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.315837] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.334602] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.353339] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.372009] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.390701] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.409418] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.428176] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.447067] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.465810] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0002-osc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.484303] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.503003] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.521673] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.540360] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.559119] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.577778] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.596473] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.615263] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.633920] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.652704] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.671519] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0003-osc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.690225] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.709008] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.727708] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.746357] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.764928] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.783410] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.801892] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.820461] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.838838] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.857376] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.875995] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0004-osc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.894430] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.912966] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.931523] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.950427] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.969019] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8961.987562] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.006149] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.024730] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.043206] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.062057] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.080474] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0005-osc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.098881] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.dirty_age_limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.117202] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.dump_stack_on_error | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.135651] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.early_lock_cancel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.154116] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.lru_cancel_batch | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.172502] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.lru_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.190824] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.lru_size | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.209061] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.ns_recalc_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.227475] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.pool.limit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.245825] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.pool.lock_volume_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.264265] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.pool.recalc_period | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.282652] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.namespaces.lustre-OST0006-osc-ffff988c67e2a000.pool.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.301162] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.high_priority_ratio | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.319246] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.threads_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.337168] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.threads_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.355063] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_delay_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.372953] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_delay_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.390846] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_delay_pct | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.408647] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.nrs_policies | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.426583] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.req_buffer_history_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.444669] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.req_buffers_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.462763] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.services.ldlm_cbd.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.480523] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.dump_granted_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.498177] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ldlm.dump_namespaces | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.515931] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.checksum_pages | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.533957] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.552079] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.default_easize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.570454] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.enable_filename_encryption | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.588626] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.enable_setstripe_gid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.606769] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.enable_statahead_fname | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.624936] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.fast_read | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.642885] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.file_heat | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.660880] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.filename_enc_use_old_base64 | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.679203] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.foreign_symlink_enable | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.697262] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.foreign_symlink_prefix | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.715451] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.foreign_symlink_upcall | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.733622] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.foreign_symlink_upcall_info | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.751800] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.heat_decay_percentage | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.770086] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.heat_period_second | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.788452] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.hybrid_io | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.806518] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.hybrid_io_read_threshold_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.825017] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.hybrid_io_write_threshold_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.843462] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.inode_cache | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.861603] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.intent_mkdir | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.879737] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lazystatfs | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.897963] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.qos_maxage | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.916305] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.qos_prio_free | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.934523] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.qos_threshold_rr | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.952919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.971735] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8962.990619] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.009417] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.028237] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.047160] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.066494] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.085306] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.104254] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.123347] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.max_mod_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.142319] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.161160] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lmv.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.180285] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.stripecount | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.198556] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.stripeoffset | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.216761] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.stripesize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.234782] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.stripetype | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.252962] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.271711] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.290497] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.309129] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.327816] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.346845] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.365695] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.384891] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.403789] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.422888] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.441901] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.460963] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.480085] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.499061] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.518176] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.537313] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.556352] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0000-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.575496] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.594439] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.613588] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.632824] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.652114] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.671604] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.690885] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.710523] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.729664] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.748990] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.768249] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.787390] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.806347] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.825412] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.844718] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.864090] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.883587] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0001-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.902896] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.922090] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.941263] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.960477] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.979617] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8963.998865] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.018028] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.037283] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.056475] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.075730] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.094725] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.113798] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.133053] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.152113] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.171325] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.190446] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.209597] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0002-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.228669] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.247748] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.266748] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.285641] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.304725] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.323756] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.342747] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.361941] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.380963] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.400073] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.419048] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.438004] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.457066] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.476017] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.495267] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.514292] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.533298] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0003-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.552347] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.571227] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.590080] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.608879] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.627742] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.646658] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.665468] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.684569] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.703464] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.722515] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.741368] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.760284] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.779370] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.798347] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.817336] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.836281] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.855200] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0004-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.874158] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.893110] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.912118] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.931104] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.950254] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.969302] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8964.989053] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.008319] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.027386] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.046584] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.065602] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.084626] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.103732] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.122834] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.141927] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.161096] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.180132] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0005-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.199192] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.218163] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.237122] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.256097] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.275149] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.294232] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.313315] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.332513] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.351566] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.370963] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.390000] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.409036] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.428207] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.447254] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.466301] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.485343] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.504305] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.lov.target_obds.lustre-OST0006-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.523362] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.max_read_ahead_async_active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.541898] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.max_read_ahead_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.560302] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.max_read_ahead_per_file_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.578765] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.max_read_ahead_whole_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.597144] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.opencache_max_ms | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.615490] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.opencache_threshold_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.633876] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.opencache_threshold_ms | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.652373] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.parallel_dio | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.670545] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.pcc_async_affinity | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.688956] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.pcc_async_threshold | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.707342] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.pcc_mode | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.725544] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.read_ahead_async_file_threshold_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.744101] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.read_ahead_range_kb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.762307] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.stat_blocksize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.780525] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_agl | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.798814] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_batch_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.819877] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_fname_match_hit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.838322] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_fname_predict_hit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.856809] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.875077] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.893308] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_running_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.911744] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.930157] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statfs_max_age | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.948510] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statfs_project | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.966866] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.stats_track_gid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8965.985124] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.stats_track_pid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.003447] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.stats_track_ppid | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.021755] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.tiny_write | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.039995] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.unaligned_dio | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.058389] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.xattr_cache | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.076745] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.enable_mlock_pages | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.095179] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.extents_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.113437] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.extents_stats_per_process | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.131927] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.max_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.150191] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.nosquash_nids | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.168453] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.offset_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.186631] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.pcc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.204640] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.read_ahead_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.222877] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.root_squash | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.240999] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.statahead_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.259197] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.277197] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.295506] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param llite.lustre-ffff988c67e2a000.unstable_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.313693] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.qos_maxage | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.331889] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.qos_prio_free | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.350566] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.qos_threshold_rr | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.368924] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.387815] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.406814] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.425716] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.444625] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.463841] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.482893] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.501883] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.520875] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.540045] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.max_mod_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.559288] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.578335] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.target_obds.lustre-MDT0000-mdc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.597339] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.md_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.615572] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lmv.lustre-clilmv-ffff988c67e2a000.qos_exclude_prefixes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.634024] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.stripecount | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.652371] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.stripeoffset | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.670688] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.stripesize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.688914] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.stripetype | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.707319] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.726239] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.745209] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.764162] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.783259] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.802216] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.821129] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.840459] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.859525] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.878663] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.897715] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.916865] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.935950] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.955026] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.974179] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8966.993189] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.012374] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0000-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.031457] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.050296] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.069460] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.088352] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.107306] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.126331] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.145240] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.164493] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.183452] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.202643] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.221628] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.240714] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.259834] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.278815] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.297974] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.316940] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.336022] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0001-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.355189] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.374262] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.393186] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.412155] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.431290] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.450208] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.469108] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.488348] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.507498] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.526728] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.545733] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.564738] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.583942] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.603060] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.622158] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.641228] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.660271] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0002-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.679478] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.698363] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.717349] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.736364] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.755334] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.774257] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.793146] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.812369] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.831382] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.850524] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.869509] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.888557] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.907812] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.926962] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.946138] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.965289] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8967.984417] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0003-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.003635] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.022588] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.041568] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.060660] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.079648] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.098740] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.117763] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.137040] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.156083] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.175360] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.194326] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.213277] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.232425] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.251553] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.270695] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.289751] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.308811] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0004-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.327920] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.346935] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.365991] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.384983] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.403930] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.422992] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.442027] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.461373] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.480458] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.499843] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.518849] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.537865] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.556980] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.576044] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.595241] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.614302] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.633231] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0005-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.652455] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.671564] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.690544] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.709494] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.728449] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.747498] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.766456] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.785526] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.804588] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.823831] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.842855] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.861897] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.881017] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.900092] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.919261] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.938338] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.957240] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param lov.lustre-clilov-ffff988c67e2a000.target_obds.lustre-OST0006-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.976327] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8968.994108] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.012477] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.030860] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.049235] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.067587] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.at_unhealthy_factor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.086098] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.104538] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.122850] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.141292] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.159947] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.max_mod_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.178555] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.197097] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.215684] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.234060] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.252415] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.batch_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.270771] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.289190] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.307556] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.326066] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.344620] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.md_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.363021] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.mdc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.381445] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.mdc_dom_min_repsize | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.399950] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.mdc_lsom | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.418261] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.mdc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.436496] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.454905] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mdc.lustre-MDT0000-mdc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.473257] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.22.189@tcp.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.491408] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.22.189@tcp.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.509462] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.22.189@tcp.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.527535] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.22.189@tcp.dynamic_nids | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.545707] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.22.189@tcp.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.563906] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param mgc.MGC10.240.22.189@tcp.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.582056] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.600454] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.618928] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.637185] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.655506] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.673921] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.692214] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.710864] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.729328] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.747921] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.766326] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.784572] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.803086] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.821510] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.840050] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.858627] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.877108] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.895581] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.913989] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.932340] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.950647] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.968996] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8969.987295] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.005650] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.023961] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.042219] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.060783] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.079177] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0000-osc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.097436] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.100517] Lustre: lustre-OST0000-osc-ffff988c67e2a000: disconnect after 20s idle
[ 8970.117888] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.136194] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.154349] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.172677] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.191039] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.209345] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.227781] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.246041] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.264412] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.282733] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.301151] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.319606] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.338007] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.356471] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.375912] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.394305] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.412673] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.431268] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.449521] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.468001] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.486501] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.504790] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.523424] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.541984] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.560411] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.579031] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.597459] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0001-osc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.616016] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.634340] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.652780] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.671219] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.689518] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.708008] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.726444] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.744934] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.763448] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.781983] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.800402] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.818843] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.837361] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.855862] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.874385] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.893535] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.911968] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.930366] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.948855] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.967342] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8970.985820] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.004299] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.022656] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.041895] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.060334] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.078776] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.097433] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.115904] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0002-osc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.134188] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.152476] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.170851] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.189144] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.207516] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.225940] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.244293] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.262984] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.281552] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.300191] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.318639] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.337030] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.355516] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.374060] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.392526] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.411757] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.430163] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.448568] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.467136] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.485432] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.503924] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.522271] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.540572] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.559209] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.577612] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.596065] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.614634] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.633012] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0003-osc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.651405] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.669823] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.688184] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.706428] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.724727] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.743152] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.761514] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.779993] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.798384] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.817083] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.835483] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.853923] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.872400] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.890855] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.909227] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.928280] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.946552] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.964961] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8971.983268] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.001488] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.019887] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.038537] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.057021] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.075438] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.093834] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.112122] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.130873] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.149438] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0004-osc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.167871] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.186140] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.204499] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.222824] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.240964] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.259424] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.277869] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.296366] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.314872] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.333440] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.351859] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.370319] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.388690] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.407239] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.425814] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.445088] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.463596] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.482018] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.500304] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.518525] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.536969] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.555308] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.573610] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.592080] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.610529] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.628999] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.647463] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.665755] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0005-osc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.684023] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.active | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.702440] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.at_history | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.720866] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.at_max | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.739190] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.at_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.757425] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.checksum_dump | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.775721] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.checksums | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.794330] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.enable_page_cache_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.813238] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.grant_shrink | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.831709] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.grant_shrink_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.850235] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.idle_connect | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.868632] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.idle_timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.887020] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.ldlm_enqueue_min | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.905363] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.max_dirty_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.923653] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.max_rpcs_in_flight | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.942318] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.ping | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.961478] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.resend_count | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.979806] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.short_io_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8972.998181] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.srpc_sepol | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.016643] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.034977] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.checksum_type | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.053385] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.cur_grant_bytes | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.071704] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.import | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.090193] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.max_pages_per_rpc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.108725] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.osc_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.127190] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.osc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.145585] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.osc_unevict_cached_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.164050] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.pinger_recov | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.182397] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST0006-osc-ffff988c67e2a000.rpc_stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.200919] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param ping_interval | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.218726] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param timeout | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.236515] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_backoff | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.254267] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_max_delay_centisecs | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.272238] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_min_delay_centisecs | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.290164] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param console_ratelimit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.307957] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param daemon_file | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.325574] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.343302] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_log_upcall | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.360912] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_marker | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.378559] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_mb | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.396129] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param debug_path | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.413758] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param dump_kernel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.431487] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fail_err | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.449113] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fail_loc | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.466742] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fail_val | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.484381] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param force_lbug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.501979] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param nis | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.519523] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param panic_on_lbug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.537289] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param peers | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.554870] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param portal_rotor | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.572526] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param printk | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.590040] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param stats | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.607619] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param subsystem_debug | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.625317] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param watchdog_ratelimit | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.643026] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fld.cli-lustre-clilmv-ffff988c67e2a000.cache_flush | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.661235] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param fld.cli-lustre-clilmv-ffff988c67e2a000.hash | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.679435] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-MDT0000-mdc-ffff988c67e2a000.space | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.697791] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-MDT0000-mdc-ffff988c67e2a000.width | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.716153] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilmv-ffff988c67e2a000.space | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.734391] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilmv-ffff988c67e2a000.width | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.752625] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilov-ffff988c67e2a000.space | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.770891] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param seq.cli-cli-lustre-clilov-ffff988c67e2a000.width | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.789222] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.init_channel | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.807028] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.lgss_keyring.debug_level | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.825098] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.gss_check_upcall_ns | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.843019] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.krb5_allow_old_client_csum | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.861094] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsc_info | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.878864] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_acquire_expire | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.896843] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_entry_expire | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.914812] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_info | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8973.932569] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param sptlrpc.gss.rsi_upcall | awk -v FS=\v -v RS=\v\v 'END { if(NR>0 && $NF !~ /.*\n$/) 					print FILENAME}'
[ 8990.579102] Lustre: lustre-OST0001-osc-ffff988c67e2a000: disconnect after 20s idle
[ 8990.580624] Lustre: Skipped 6 previous similar messages
[ 8995.699099] Lustre: lustre-OST0003-osc-ffff988c67e2a000: disconnect after 24s idle
[ 8995.700631] Lustre: Skipped 1 previous similar message
[ 9209.145723] Autotest: Test running for 150 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 9434.067521] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 134a: Server reclaims locks when reaching lock_reclaim_threshold ========================================================== 02:08:17 \(1734401297\)
[ 9434.436087] Lustre: DEBUG MARKER: == sanity test 134a: Server reclaims locks when reaching lock_reclaim_threshold ========================================================== 02:08:17 (1734401297)
[ 9435.171947] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9436.365501] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9449.357415] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9450.488144] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9453.112162] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 134b: Server rejects lock request when reaching lock_limit_mb ========================================================== 02:08:36 \(1734401316\)
[ 9453.477918] Lustre: DEBUG MARKER: == sanity test 134b: Server rejects lock request when reaching lock_limit_mb ========================================================== 02:08:36 (1734401316)
[ 9455.437770] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9475.873860] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9476.976327] Lustre: lustre-OST0005-osc-ffff988c67e2a000: disconnect after 23s idle
[ 9476.977902] Lustre: Skipped 3 previous similar messages
[ 9477.462090] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[ 9478.297440] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[ 9480.881078] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_135 skipping SLOW test 135
[ 9481.242232] Lustre: DEBUG MARKER: SKIP: sanity test_135 skipping SLOW test 135
[ 9481.663389] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_136 skipping SLOW test 136
[ 9482.024160] Lustre: DEBUG MARKER: SKIP: sanity test_136 skipping SLOW test 136
[ 9482.471583] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 140: Check reasonable stack depth \(shouldn\'t LBUG\) ============================================================== 02:09:05 \(1734401345\)
[ 9482.838659] Lustre: DEBUG MARKER: == sanity test 140: Check reasonable stack depth (shouldn't LBUG) ============================================================== 02:09:05 (1734401345)
[ 9487.555055] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150a: truncate\/append tests =============== 02:09:11 \(1734401351\)
[ 9487.923538] Lustre: DEBUG MARKER: == sanity test 150a: truncate/append tests =============== 02:09:11 (1734401351)
[ 9488.104933] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[ 9488.117268] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[ 9488.294352] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[ 9507.696103] Lustre: lustre-OST0004-osc-ffff988c67e2a000: disconnect after 20s idle
[ 9509.974132] Autotest: Test running for 155 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 9514.325016] Lustre: Unmounted lustre-client
[ 9514.334449] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[ 9514.346220] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[ 9514.408915] Lustre: Mounted lustre-client
[ 9525.778980] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150b: Verify fallocate \(prealloc\) functionality ========================================================== 02:09:49 \(1734401389\)
[ 9526.145459] Lustre: DEBUG MARKER: == sanity test 150b: Verify fallocate (prealloc) functionality ========================================================== 02:09:49 (1734401389)
[ 9537.564873] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150bb: Verify fallocate modes both zero space ========================================================== 02:10:01 \(1734401401\)
[ 9537.934261] Lustre: DEBUG MARKER: == sanity test 150bb: Verify fallocate modes both zero space ========================================================== 02:10:01 (1734401401)
[ 9553.639419] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150c: Verify fallocate Size and Blocks ==== 02:10:17 \(1734401417\)
[ 9554.010347] Lustre: DEBUG MARKER: == sanity test 150c: Verify fallocate Size and Blocks ==== 02:10:17 (1734401417)
[ 9568.597161] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150d: Verify fallocate Size and Blocks - Non zero start ========================================================== 02:10:32 \(1734401432\)
[ 9569.048050] Lustre: DEBUG MARKER: == sanity test 150d: Verify fallocate Size and Blocks - Non zero start ========================================================== 02:10:32 (1734401432)
[ 9589.844463] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150e: Verify 60% of available OST space consumed by fallocate ========================================================== 02:10:48 \(1734401448\)
[ 9590.290798] Lustre: DEBUG MARKER: == sanity test 150e: Verify 60% of available OST space consumed by fallocate ========================================================== 02:10:48 (1734401448)
[ 9608.156809] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150f: Verify fallocate punch functionality ========================================================== 02:11:11 \(1734401471\)
[ 9608.527160] Lustre: DEBUG MARKER: == sanity test 150f: Verify fallocate punch functionality ========================================================== 02:11:11 (1734401471)
[ 9625.209686] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150g: Verify fallocate punch on large range ========================================================== 02:11:28 \(1734401488\)
[ 9625.576631] Lustre: DEBUG MARKER: == sanity test 150g: Verify fallocate punch on large range ========================================================== 02:11:28 (1734401488)
[ 9641.322044] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 150h: Verify extend fallocate updates the file size ========================================================== 02:11:44 \(1734401504\)
[ 9641.770100] Lustre: DEBUG MARKER: == sanity test 150h: Verify extend fallocate updates the file size ========================================================== 02:11:44 (1734401504)
[ 9644.030572] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 151: test cache on oss and controls ========================================================================================= 02:11:47 \(1734401507\)
[ 9644.397508] Lustre: DEBUG MARKER: == sanity test 151: test cache on oss and controls ========================================================================================= 02:11:47 (1734401507)
[ 9656.787864] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 152: test read\/write with enomem ====================================================================================== 02:12:00 \(1734401520\)
[ 9657.155991] Lustre: DEBUG MARKER: == sanity test 152: test read/write with enomem ====================================================================================== 02:12:00 (1734401520)
[ 9660.235679] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 153: test if fdatasync does not crash ================================================================================= 02:12:03 \(1734401523\)
[ 9660.603575] Lustre: DEBUG MARKER: == sanity test 153: test if fdatasync does not crash ================================================================================= 02:12:03 (1734401523)
[ 9662.807608] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154A: lfs path2fid and fid2path basic checks ========================================================== 02:12:06 \(1734401526\)
[ 9663.181679] Lustre: DEBUG MARKER: == sanity test 154A: lfs path2fid and fid2path basic checks ========================================================== 02:12:06 (1734401526)
[ 9665.103192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154B: verify the ll_decode_linkea tool ==== 02:12:08 \(1734401528\)
[ 9665.472195] Lustre: DEBUG MARKER: == sanity test 154B: verify the ll_decode_linkea tool ==== 02:12:08 (1734401528)
[ 9667.407092] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154a: Open-by-FID ========================= 02:12:10 \(1734401530\)
[ 9667.777808] Lustre: DEBUG MARKER: == sanity test 154a: Open-by-FID ========================= 02:12:10 (1734401530)
[ 9670.032680] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154b: Open-by-FID for remote directory ==== 02:12:13 \(1734401533\)
[ 9670.401118] Lustre: DEBUG MARKER: == sanity test 154b: Open-by-FID for remote directory ==== 02:12:13 (1734401533)
[ 9670.819188] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_154b needs \>= 2 MDTs
[ 9671.184436] Lustre: DEBUG MARKER: SKIP: sanity test_154b needs >= 2 MDTs
[ 9671.635227] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154c: lfs path2fid and fid2path multiple arguments ========================================================== 02:12:15 \(1734401535\)
[ 9672.001629] Lustre: DEBUG MARKER: == sanity test 154c: lfs path2fid and fid2path multiple arguments ========================================================== 02:12:15 (1734401535)
[ 9673.915326] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154d: Verify open file fid ================ 02:12:17 \(1734401537\)
[ 9674.278067] Lustre: DEBUG MARKER: == sanity test 154d: Verify open file fid ================ 02:12:17 (1734401537)
[ 9676.558279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154e: .lustre is not returned by readdir == 02:12:20 \(1734401540\)
[ 9676.923779] Lustre: DEBUG MARKER: == sanity test 154e: .lustre is not returned by readdir == 02:12:20 (1734401540)
[ 9678.805663] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154ea: .lustre is not returned by readdir \(2\) ========================================================== 02:12:22 \(1734401542\)
[ 9679.169709] Lustre: DEBUG MARKER: == sanity test 154ea: .lustre is not returned by readdir (2) ========================================================== 02:12:22 (1734401542)
[ 9700.335282] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154f: get parent fids by reading link ea == 02:12:43 \(1734401563\)
[ 9701.206661] Lustre: DEBUG MARKER: == sanity test 154f: get parent fids by reading link ea == 02:12:43 (1734401563)
[ 9701.377459] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[ 9701.377459] 			 [[ -z "" ]] && param= ||
[ 9701.377459] 			 param=$(grep  <<< "$params");
[ 9701.377459] 			 [[ -z $param ]] && param="$params";
[ 9701.377459] 			 while read s; do echo client $s;
[ 9701.377459] 			 done <<< "$param"
[ 9701.451653] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff988c01422800.xattr_cache=1
[ 9703.331989] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154g: various llapi FID tests ============= 02:12:46 \(1734401566\)
[ 9703.700821] Lustre: DEBUG MARKER: == sanity test 154g: various llapi FID tests ============= 02:12:46 (1734401566)
[ 9810.849635] Autotest: Test running for 160 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[ 9845.325378] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154h: Verify interactive path2fid ========= 02:15:08 \(1734401708\)
[ 9845.693904] Lustre: DEBUG MARKER: == sanity test 154h: Verify interactive path2fid ========= 02:15:08 (1734401708)
[ 9848.036970] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 154i: fid2path for path longer than PATH_MAX ========================================================== 02:15:11 \(1734401711\)
[ 9848.403968] Lustre: DEBUG MARKER: == sanity test 154i: fid2path for path longer than PATH_MAX ========================================================== 02:15:11 (1734401711)
[ 9853.575783] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155a: Verify small file correctness: read cache:on write_cache:on ========================================================== 02:15:17 \(1734401717\)
[ 9853.944336] Lustre: DEBUG MARKER: == sanity test 155a: Verify small file correctness: read cache:on write_cache:on ========================================================== 02:15:17 (1734401717)
[ 9861.228424] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155b: Verify small file correctness: read cache:on write_cache:off ========================================================== 02:15:24 \(1734401724\)
[ 9861.602102] Lustre: DEBUG MARKER: == sanity test 155b: Verify small file correctness: read cache:on write_cache:off ========================================================== 02:15:24 (1734401724)
[ 9868.827273] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155c: Verify small file correctness: read cache:off write_cache:on ========================================================== 02:15:32 \(1734401732\)
[ 9869.193758] Lustre: DEBUG MARKER: == sanity test 155c: Verify small file correctness: read cache:off write_cache:on ========================================================== 02:15:32 (1734401732)
[ 9876.496120] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155d: Verify small file correctness: read cache:off write_cache:off ========================================================== 02:15:39 \(1734401739\)
[ 9876.866870] Lustre: DEBUG MARKER: == sanity test 155d: Verify small file correctness: read cache:off write_cache:off ========================================================== 02:15:39 (1734401739)
[ 9884.188320] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155e: Verify big file correctness: read cache:on write_cache:on ========================================================== 02:15:47 \(1734401747\)
[ 9884.556270] Lustre: DEBUG MARKER: == sanity test 155e: Verify big file correctness: read cache:on write_cache:on ========================================================== 02:15:47 (1734401747)
[ 9895.731506] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155f: Verify big file correctness: read cache:on write_cache:off ========================================================== 02:15:59 \(1734401759\)
[ 9896.100313] Lustre: DEBUG MARKER: == sanity test 155f: Verify big file correctness: read cache:on write_cache:off ========================================================== 02:15:59 (1734401759)
[ 9906.465747] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155g: Verify big file correctness: read cache:off write_cache:on ========================================================== 02:16:09 \(1734401769\)
[ 9906.832448] Lustre: DEBUG MARKER: == sanity test 155g: Verify big file correctness: read cache:off write_cache:on ========================================================== 02:16:09 (1734401769)
[ 9917.223035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 155h: Verify big file correctness: read cache:off write_cache:off ========================================================== 02:16:20 \(1734401780\)
[ 9917.591582] Lustre: DEBUG MARKER: == sanity test 155h: Verify big file correctness: read cache:off write_cache:off ========================================================== 02:16:20 (1734401780)
[ 9927.942123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 156: Verification of tunables ============= 02:16:31 \(1734401791\)
[ 9928.309725] Lustre: DEBUG MARKER: == sanity test 156: Verification of tunables ============= 02:16:31 (1734401791)
[ 9936.336337] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn on read and write cache
[ 9936.698101] Lustre: DEBUG MARKER: Turn on read and write cache
[ 9937.926457] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back.
[ 9938.288414] Lustre: DEBUG MARKER: Write data and read it back.
[ 9938.705636] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read should be satisfied from the cache.
[ 9939.064718] Lustre: DEBUG MARKER: Read should be satisfied from the cache.
[ 9940.322279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits: before: 8277, after: 8280
[ 9940.683543] Lustre: DEBUG MARKER: cache hits: before: 8277, after: 8280
[ 9941.102432] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read again\; it should be satisfied from the cache.
[ 9941.464531] Lustre: DEBUG MARKER: Read again; it should be satisfied from the cache.
[ 9942.306002] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8280, after: 8283
[ 9942.670015] Lustre: DEBUG MARKER: cache hits:: before: 8280, after: 8283
[ 9943.087182] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn off the read cache and turn on the write cache
[ 9943.447923] Lustre: DEBUG MARKER: Turn off the read cache and turn on the write cache
[ 9944.677340] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read again\; it should be satisfied from the cache.
[ 9945.042168] Lustre: DEBUG MARKER: Read again; it should be satisfied from the cache.
[ 9946.294921] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8283, after: 8286
[ 9946.658800] Lustre: DEBUG MARKER: cache hits:: before: 8283, after: 8286
[ 9947.085038] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back.
[ 9947.451313] Lustre: DEBUG MARKER: Write data and read it back.
[ 9947.866489] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read should be satisfied from the cache.
[ 9948.229842] Lustre: DEBUG MARKER: Read should be satisfied from the cache.
[ 9949.492832] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8286, after: 8289
[ 9949.861522] Lustre: DEBUG MARKER: cache hits:: before: 8286, after: 8289
[ 9950.279893] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn off read and write cache
[ 9950.642876] Lustre: DEBUG MARKER: Turn off read and write cache
[ 9951.887584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back
[ 9952.251578] Lustre: DEBUG MARKER: Write data and read it back
[ 9952.668122] Lustre: DEBUG MARKER: /usr/sbin/lctl mark It should not be satisfied from the cache.
[ 9953.033541] Lustre: DEBUG MARKER: It should not be satisfied from the cache.
[ 9954.306567] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8289, after: 8289
[ 9954.672457] Lustre: DEBUG MARKER: cache hits:: before: 8289, after: 8289
[ 9955.089169] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Turn on the read cache and turn off the write cache
[ 9955.452386] Lustre: DEBUG MARKER: Turn on the read cache and turn off the write cache
[ 9956.711666] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Write data and read it back
[ 9957.074862] Lustre: DEBUG MARKER: Write data and read it back
[ 9957.502977] Lustre: DEBUG MARKER: /usr/sbin/lctl mark It should not be satisfied from the cache.
[ 9957.867014] Lustre: DEBUG MARKER: It should not be satisfied from the cache.
[ 9959.166555] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8289, after: 8289
[ 9959.530300] Lustre: DEBUG MARKER: cache hits:: before: 8289, after: 8289
[ 9959.948474] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Read again\; it should be satisfied from the cache.
[ 9960.311617] Lustre: DEBUG MARKER: Read again; it should be satisfied from the cache.
[ 9961.568733] Lustre: DEBUG MARKER: /usr/sbin/lctl mark cache hits:: before: 8289, after: 8292
[ 9961.935720] Lustre: DEBUG MARKER: cache hits:: before: 8289, after: 8292
[ 9965.960317] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160a: changelog sanity ==================== 02:17:09 \(1734401829\)
[ 9966.322769] Lustre: DEBUG MARKER: == sanity test 160a: changelog sanity ==================== 02:17:09 (1734401829)
[ 9975.150664] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[ 9975.153787] Lustre: lustre-MDT0000-mdc-ffff988c01422800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[ 9975.157048] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f0e2d34e to 0xdf05ec28f0ee3236
[ 9975.159676] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[ 9976.700059] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988c2f0fad80 x1818650996591104/t30064776247(30064776247) o101->lustre-MDT0000-mdc-ffff988c01422800@10.240.22.189@tcp:12/10 lens 984/608 e 0 to 0 dl 1734401856 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'rm.0' uid:0 gid:0
[ 9980.279429] Lustre: lustre-MDT0000-mdc-ffff988c01422800: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[ 9984.024445] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160b: Verify that very long rename doesn\'t crash in changelog ========================================================== 02:17:27 \(1734401847\)
[ 9984.391788] Lustre: DEBUG MARKER: == sanity test 160b: Verify that very long rename doesn't crash in changelog ========================================================== 02:17:27 (1734401847)
[ 9988.789031] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160c: verify that changelog log catch the truncate event ========================================================== 02:17:32 \(1734401852\)
[ 9989.155580] Lustre: DEBUG MARKER: == sanity test 160c: verify that changelog log catch the truncate event ========================================================== 02:17:32 (1734401852)
[ 9993.732257] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160d: verify that changelog log catch the migrate event ========================================================== 02:17:37 \(1734401857\)
[ 9994.094252] Lustre: DEBUG MARKER: == sanity test 160d: verify that changelog log catch the migrate event ========================================================== 02:17:37 (1734401857)
[ 9994.552052] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_160d needs \>= 2 MDTs
[ 9994.918525] Lustre: DEBUG MARKER: SKIP: sanity test_160d needs >= 2 MDTs
[ 9995.374459] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160e: changelog negative testing \(should return errors\) ========================================================== 02:17:38 \(1734401858\)
[ 9995.742749] Lustre: DEBUG MARKER: == sanity test 160e: changelog negative testing (should return errors) ========================================================== 02:17:38 (1734401858)
[10000.116190] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160f: changelog garbage collect \(timestamped users\) ========================================================== 02:17:43 \(1734401863\)
[10000.480000] Lustre: DEBUG MARKER: == sanity test 160f: changelog garbage collect (timestamped users) ========================================================== 02:17:43 (1734401863)
[10002.848763] Lustre: DEBUG MARKER: /usr/sbin/lctl mark 1734401866: creating first files
[10003.217208] Lustre: DEBUG MARKER: 1734401866: creating first files
[10020.677297] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160g: changelog garbage collect on idle records ========================================================== 02:18:04 \(1734401884\)
[10021.042417] Lustre: DEBUG MARKER: == sanity test 160g: changelog garbage collect on idle records ========================================================== 02:18:04 (1734401884)
[10032.002127] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160h: changelog gc thread stop upon umount, orphan records delete ========================================================== 02:18:15 \(1734401895\)
[10032.370238] Lustre: DEBUG MARKER: == sanity test 160h: changelog gc thread stop upon umount, orphan records delete ========================================================== 02:18:15 (1734401895)
[10057.070365] Lustre: lustre-MDT0000-mdc-ffff988c01422800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10057.073736] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[10057.077313] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f0ee3236 to 0xdf05ec28f0ee4530
[10057.079740] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[10057.081631] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988c2f0fad80 x1818650996591104/t30064776247(30064776247) o101->lustre-MDT0000-mdc-ffff988c01422800@10.240.22.189@tcp:12/10 lens 984/608 e 0 to 0 dl 1734401936 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'rm.0' uid:0 gid:0
[10057.086644] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 76 previous similar messages
[10062.915930] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160i: changelog user register\/unregister race ========================================================== 02:18:46 \(1734401926\)
[10063.281912] Lustre: DEBUG MARKER: == sanity test 160i: changelog user register/unregister race ========================================================== 02:18:46 (1734401926)
[10067.312795] Lustre: 432370:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734401915/real 1734401915]  req@ffff988c010a96c0 x1818651005883520/t0(0) o400->lustre-MDT0000-mdc-ffff988c01422800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734401931 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[10072.241882] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160j: client can be umounted while its chanangelog is being used ========================================================== 02:18:55 \(1734401935\)
[10072.608357] Lustre: DEBUG MARKER: == sanity test 160j: client can be umounted while its chanangelog is being used ========================================================== 02:18:55 (1734401935)
[10072.658075] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre2
[10072.670092] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre2
[10072.718420] Lustre: Mounted lustre-client
[10073.814684] Lustre: Unmounted lustre-client
[10075.569688] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[10075.581609] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[10075.626908] Lustre: Mounted lustre-client
[10076.961096] Lustre: Unmounted lustre-client
[10077.395921] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160k: Verify that changelog records are not lost ========================================================== 02:19:00 \(1734401940\)
[10077.761213] Lustre: DEBUG MARKER: == sanity test 160k: Verify that changelog records are not lost ========================================================== 02:19:00 (1734401940)
[10089.234371] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160l: Verify that MTIME changelog records contain the parent FID ========================================================== 02:19:12 \(1734401952\)
[10089.602400] Lustre: DEBUG MARKER: == sanity test 160l: Verify that MTIME changelog records contain the parent FID ========================================================== 02:19:12 (1734401952)
[10095.924962] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160m: Changelog clear race ================ 02:19:19 \(1734401959\)
[10096.291241] Lustre: DEBUG MARKER: == sanity test 160m: Changelog clear race ================ 02:19:19 (1734401959)
[10104.776192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160n: Changelog destroy race ============== 02:19:28 \(1734401968\)
[10105.141663] Lustre: DEBUG MARKER: == sanity test 160n: Changelog destroy race ============== 02:19:28 (1734401968)
[10107.106138] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10110.181232] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10111.886080] Autotest: Test running for 165 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[10161.229243] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10166.058427] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10167.785275] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10170.912968] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10222.413527] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10227.398756] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10229.159203] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10232.242806] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10283.586283] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10288.623401] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10296.419035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160o: changelog user name and mask ======== 02:22:39 \(1734402159\)
[10296.790179] Lustre: DEBUG MARKER: == sanity test 160o: changelog user name and mask ======== 02:22:39 (1734402159)
[10306.305388] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160p: Changelog orphan cleanup with no users ========================================================== 02:22:49 \(1734402169\)
[10306.672738] Lustre: DEBUG MARKER: == sanity test 160p: Changelog orphan cleanup with no users ========================================================== 02:22:49 (1734402169)
[10314.991695] LustreError: lustre-MDT0000-mdc-ffff988c0b808800: operation llog_origin_handle_open to node 10.240.22.189@tcp failed: rc = -107
[10314.994086] Lustre: lustre-MDT0000-mdc-ffff988c0b808800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10315.003802] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988c7652e3c0 x1818651005967104/t38654705792(38654705792) o101->lustre-MDT0000-mdc-ffff988c0b808800@10.240.22.189@tcp:12/10 lens 576/608 e 0 to 0 dl 1734402194 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'lfs.0' uid:0 gid:0
[10315.008943] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 79 previous similar messages
[10315.015082] Lustre: lustre-MDT0000-mdc-ffff988c0b808800: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[10315.017082] Lustre: Skipped 1 previous similar message
[10316.396457] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[10316.399768] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f0ee4530 to 0xdf05ec28f131bb17
[10317.516827] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160q: changelog effective mask is DEFMASK if not set ========================================================== 02:23:00 \(1734402180\)
[10317.891137] Lustre: DEBUG MARKER: == sanity test 160q: changelog effective mask is DEFMASK if not set ========================================================== 02:23:00 (1734402180)
[10321.110875] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160s: changelog garbage collect on idle records \* time ========================================================== 02:23:04 \(1734402184\)
[10321.479492] Lustre: DEBUG MARKER: == sanity test 160s: changelog garbage collect on idle records * time ========================================================== 02:23:04 (1734402184)
[10327.659313] Lustre: 432371:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734402175/real 1734402175]  req@ffff988c67438000 x1818651040617344/t0(0) o400->lustre-MDT0000-mdc-ffff988c0b808800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734402191 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[10332.771033] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160t: changelog garbage collect on lack of space ========================================================== 02:23:16 \(1734402196\)
[10333.137417] Lustre: DEBUG MARKER: == sanity test 160t: changelog garbage collect on lack of space ========================================================== 02:23:16 (1734402196)
[10334.809514] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10336.676078] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10339.362218] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10340.349920] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10350.543114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 160u: changelog rename record type name and sname strings are correct ========================================================== 02:23:34 \(1734402214\)
[10350.905976] Lustre: DEBUG MARKER: == sanity test 160u: changelog rename record type name and sname strings are correct ========================================================== 02:23:34 (1734402214)
[10355.211517] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161a: link ea sanity ====================== 02:23:38 \(1734402218\)
[10355.578894] Lustre: DEBUG MARKER: == sanity test 161a: link ea sanity ====================== 02:23:38 (1734402218)
[10356.370022] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10357.498323] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10360.389345] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10361.455236] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10362.656355] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161b: link ea sanity under remote directory ========================================================== 02:23:46 \(1734402226\)
[10363.028810] Lustre: DEBUG MARKER: == sanity test 161b: link ea sanity under remote directory ========================================================== 02:23:46 (1734402226)
[10363.448286] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_161b skipping remote directory test
[10363.816097] Lustre: DEBUG MARKER: SKIP: sanity test_161b skipping remote directory test
[10364.268109] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161c: check CL_RENME[UNLINK] changelog record flags ========================================================== 02:23:47 \(1734402227\)
[10364.635905] Lustre: DEBUG MARKER: == sanity test 161c: check CL_RENME[UNLINK] changelog record flags ========================================================== 02:23:47 (1734402227)
[10369.168854] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 161d: create with concurrent .lustre\/fid access ========================================================== 02:23:52 \(1734402232\)
[10369.543925] Lustre: DEBUG MARKER: == sanity test 161d: create with concurrent .lustre/fid access ========================================================== 02:23:52 (1734402232)
[10370.541481] LustreError: 551039:0:(namei.c:1529:ll_create_node()) cfs_fail_timeout id 140c sleeping for 5000ms
[10372.627030] LustreError: 551039:0:(namei.c:1529:ll_create_node()) cfs_fail_timeout interrupted
[10375.501043] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 162a: path lookup sanity ================== 02:23:58 \(1734402238\)
[10375.866841] Lustre: DEBUG MARKER: == sanity test 162a: path lookup sanity ================== 02:23:58 (1734402238)
[10378.166421] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 162b: striped directory path lookup sanity ========================================================== 02:24:01 \(1734402241\)
[10378.533965] Lustre: DEBUG MARKER: == sanity test 162b: striped directory path lookup sanity ========================================================== 02:24:01 (1734402241)
[10378.956890] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_162b needs \>= 2 MDTs
[10379.327972] Lustre: DEBUG MARKER: SKIP: sanity test_162b needs >= 2 MDTs
[10379.783364] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 162c: fid2path works with paths 100 or more directories deep ========================================================== 02:24:03 \(1734402243\)
[10380.171351] Lustre: DEBUG MARKER: == sanity test 162c: fid2path works with paths 100 or more directories deep ========================================================== 02:24:03 (1734402243)
[10386.395206] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165a: ofd access log discovery ============ 02:24:09 \(1734402249\)
[10386.766414] Lustre: DEBUG MARKER: == sanity test 165a: ofd access log discovery ============ 02:24:09 (1734402249)
[10398.317109] Lustre: lustre-OST0000-osc-ffff988c0b808800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10404.162226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165b: ofd access log entries are produced and consumed ========================================================== 02:24:27 \(1734402267\)
[10404.530925] Lustre: DEBUG MARKER: == sanity test 165b: ofd access log entries are produced and consumed ========================================================== 02:24:27 (1734402267)
[10412.778932] Autotest: Test running for 170 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[10434.157543] Lustre: lustre-OST0000-osc-ffff988c0b808800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10435.032154] Lustre: lustre-OST0000-osc-ffff988c0b808800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[10435.034232] Lustre: Skipped 1 previous similar message
[10437.654223] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165c: full ofd access logs do not block IOs ========================================================== 02:25:01 \(1734402301\)
[10438.019817] Lustre: DEBUG MARKER: == sanity test 165c: full ofd access logs do not block IOs ========================================================== 02:25:01 (1734402301)
[10446.126948] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10446.847240] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10454.159897] Lustre: lustre-OST0000-osc-ffff988c0b808800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10454.673215] Lustre: lustre-OST0000-osc-ffff988c0b808800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[10457.525748] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165d: ofd_access_log mask works =========== 02:25:21 \(1734402321\)
[10457.894771] Lustre: DEBUG MARKER: == sanity test 165d: ofd_access_log mask works =========== 02:25:21 (1734402321)
[10485.355595] Lustre: lustre-OST0000-osc-ffff988c0b808800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10487.411376] Lustre: lustre-OST0000-osc-ffff988c0b808800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[10490.422318] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165e: ofd_access_log MDT index filter works ========================================================== 02:25:53 \(1734402353\)
[10490.789616] Lustre: DEBUG MARKER: == sanity test 165e: ofd_access_log MDT index filter works ========================================================== 02:25:53 (1734402353)
[10491.223992] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_165e needs \>= 2 MDTs
[10491.591495] Lustre: DEBUG MARKER: SKIP: sanity test_165e needs >= 2 MDTs
[10492.053290] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 165f: ofd_access_log_reader --exit-on-close works ========================================================== 02:25:55 \(1734402355\)
[10492.425701] Lustre: DEBUG MARKER: == sanity test 165f: ofd_access_log_reader --exit-on-close works ========================================================== 02:25:55 (1734402355)
[10508.928298] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 169: parallel read and truncate should not deadlock ========================================================== 02:26:12 \(1734402372\)
[10509.303189] Lustre: DEBUG MARKER: == sanity test 169: parallel read and truncate should not deadlock ========================================================== 02:26:12 (1734402372)
[10509.726593] Lustre: DEBUG MARKER: /usr/sbin/lctl mark creating a 10 Mb file
[10510.094809] Lustre: DEBUG MARKER: creating a 10 Mb file
[10522.348972] Lustre: DEBUG MARKER: /usr/sbin/lctl mark starting reads
[10522.806194] Lustre: DEBUG MARKER: starting reads
[10523.297518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark truncating the file
[10523.666482] Lustre: DEBUG MARKER: truncating the file
[10524.177015] Lustre: DEBUG MARKER: /usr/sbin/lctl mark killing dd
[10524.540714] Lustre: DEBUG MARKER: killing dd
[10524.961743] Lustre: DEBUG MARKER: /usr/sbin/lctl mark removing the temporary file
[10525.324519] Lustre: DEBUG MARKER: removing the temporary file
[10527.206095] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 170: test lctl df to handle corrupted log =============================================================================== 02:26:30 \(1734402390\)
[10527.576745] Lustre: DEBUG MARKER: == sanity test 170: test lctl df to handle corrupted log =============================================================================== 02:26:30 (1734402390)
[10527.594936] Lustre: debug daemon will attempt to start writing to /tmp/f170.sanity_log_good (512000kB max)
[10527.640244] Lustre: shutting down debug daemon thread...
[10527.647828] Lustre: debug daemon will attempt to start writing to /tmp/f170.sanity_log_good (512000kB max)
[10527.658800] Lustre: shutting down debug daemon thread...
[10529.930080] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 171: test libcfs_debug_dumplog_thread stuck in do_exit\(\) ================================================================ 02:26:33 \(1734402393\)
[10530.296437] Lustre: DEBUG MARKER: == sanity test 171: test libcfs_debug_dumplog_thread stuck in do_exit() ================================================================ 02:26:33 (1734402393)
[10530.327951] LustreError: 560458:0:(file.c:457:ll_file_release()) cfs_fail_timeout id 50e sleeping for 3000ms
[10533.342101] LustreError: 560458:0:(file.c:457:ll_file_release()) cfs_fail_timeout id 50e awake
[10533.343907] LustreError: dumping log to /tmp/lustre-log.1734402397.560458
[10533.529719] Lustre: *** cfs_fail_loc=50e, val=3000***
[10535.523738] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 172: manual device removal with lctl cleanup\/detach ================================================================ 02:26:39 \(1734402399\)
[10535.891830] Lustre: DEBUG MARKER: == sanity test 172: manual device removal with lctl cleanup/detach ================================================================ 02:26:39 (1734402399)
[10537.272297] Lustre: *** cfs_fail_loc=60e, val=0***
[10537.275566] Lustre: Unmounted lustre-client
[10538.764815] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[10538.776304] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[10538.829823] Lustre: Mounted lustre-client
[10539.296123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 180a: test obdecho on osc ================= 02:26:42 \(1734402402\)
[10539.668047] Lustre: DEBUG MARKER: == sanity test 180a: test obdecho on osc ================= 02:26:42 (1734402402)
[10540.086404] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_180a obdecho on osc is no longer supported
[10540.452856] Lustre: DEBUG MARKER: SKIP: sanity test_180a obdecho on osc is no longer supported
[10540.906513] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 180b: test obdecho directly on obdfilter == 02:26:44 \(1734402404\)
[10541.272785] Lustre: DEBUG MARKER: == sanity test 180b: test obdecho directly on obdfilter == 02:26:44 (1734402404)
[10548.948544] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 180c: test huge bulk I\/O size on obdfilter, don\'t LASSERT ========================================================== 02:26:52 \(1734402412\)
[10549.315155] Lustre: DEBUG MARKER: == sanity test 180c: test huge bulk I/O size on obdfilter, don't LASSERT ========================================================== 02:26:52 (1734402412)
[10558.065222] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 181: Test open-unlinked dir ================================================================================== 02:27:01 \(1734402421\)
[10558.433382] Lustre: DEBUG MARKER: == sanity test 181: Test open-unlinked dir ================================================================================== 02:27:01 (1734402421)
[10559.126967] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10561.709014] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10563.157842] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10564.838256] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10567.499379] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 182a: Test parallel modify metadata operations from mdc ========================================================== 02:27:10 \(1734402430\)
[10567.867601] Lustre: DEBUG MARKER: == sanity test 182a: Test parallel modify metadata operations from mdc ========================================================== 02:27:10 (1734402430)
[10569.586783] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10570.372247] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10570.995874] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10571.355111] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10571.399177] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10571.408152] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10571.696455] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10571.737321] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10571.780751] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10571.805964] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10574.928105] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10575.205622] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10575.262718] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10575.707725] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10575.781384] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10576.491208] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10576.508682] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10577.053708] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10577.184887] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10577.399981] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10581.831590] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10581.882772] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10582.911981] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10583.263093] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10583.415877] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10583.429139] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10583.567992] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10583.780564] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10583.868147] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10583.937644] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[10586.211726] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10586.495396] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10586.593596] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10586.766446] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10587.636149] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10588.011981] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10588.028126] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10588.060265] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10588.115192] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10588.600391] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[10594.426185] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 02:27:37 \(1734402457\)
[10594.791422] Lustre: DEBUG MARKER: == sanity test 182b: Test parallel modify metadata operations from osp ========================================================== 02:27:37 (1734402457)
[10595.214825] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_182b needs \>= 2 MDTs
[10595.581631] Lustre: DEBUG MARKER: SKIP: sanity test_182b needs >= 2 MDTs
[10596.031871] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 183: No crash or request leak in case of strange dispositions ================================================================== 02:27:39 \(1734402459\)
[10596.400606] Lustre: DEBUG MARKER: == sanity test 183: No crash or request leak in case of strange dispositions ================================================================== 02:27:39 (1734402459)
[10598.922831] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184a: Basic layout swap =================== 02:27:42 \(1734402462\)
[10599.285016] Lustre: DEBUG MARKER: == sanity test 184a: Basic layout swap =================== 02:27:42 (1734402462)
[10602.319455] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184b: Forbidden layout swap \(will generate errors\) ========================================================== 02:27:45 \(1734402465\)
[10602.686467] Lustre: DEBUG MARKER: == sanity test 184b: Forbidden layout swap (will generate errors) ========================================================== 02:27:45 (1734402465)
[10604.661371] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184c: Concurrent write and layout swap ==== 02:27:48 \(1734402468\)
[10605.028941] Lustre: DEBUG MARKER: == sanity test 184c: Concurrent write and layout swap ==== 02:27:48 (1734402468)
[10623.393299] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184d: allow stripeless layouts swap ======= 02:28:06 \(1734402486\)
[10623.760093] Lustre: DEBUG MARKER: == sanity test 184d: allow stripeless layouts swap ======= 02:28:06 (1734402486)
[10628.645691] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184e: Recreate layout after stripeless layout swaps ========================================================== 02:28:12 \(1734402492\)
[10629.011641] Lustre: DEBUG MARKER: == sanity test 184e: Recreate layout after stripeless layout swaps ========================================================== 02:28:12 (1734402492)
[10633.775206] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 184f: IOC_MDC_GETFILEINFO for files with long names but no striping ========================================================== 02:28:17 \(1734402497\)
[10634.147762] Lustre: DEBUG MARKER: == sanity test 184f: IOC_MDC_GETFILEINFO for files with long names but no striping ========================================================== 02:28:17 (1734402497)
[10636.035293] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 185: Volatile file support ================ 02:28:19 \(1734402499\)
[10636.403184] Lustre: DEBUG MARKER: == sanity test 185: Volatile file support ================ 02:28:19 (1734402499)
[10639.325121] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 185a: Volatile file creation in .lustre\/fid\/ ========================================================== 02:28:22 \(1734402502\)
[10639.690004] Lustre: DEBUG MARKER: == sanity test 185a: Volatile file creation in .lustre/fid/ ========================================================== 02:28:22 (1734402502)
[10643.605958] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 187a: Test data version change ============ 02:28:27 \(1734402507\)
[10643.975841] Lustre: DEBUG MARKER: == sanity test 187a: Test data version change ============ 02:28:27 (1734402507)
[10646.634584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 187b: Test data version change on volatile file ========================================================== 02:28:30 \(1734402510\)
[10647.004890] Lustre: DEBUG MARKER: == sanity test 187b: Test data version change on volatile file ========================================================== 02:28:30 (1734402510)
[10649.071576] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 200: OST pools ============================ 02:28:32 \(1734402512\)
[10649.442406] Lustre: DEBUG MARKER: == sanity test 200: OST pools ============================ 02:28:32 (1734402512)
[10651.784462] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1 		2>/dev/null || echo foo
[10654.139570] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1 |
[10654.139570] 			grep -e lustre-OST0000_UUID -e lustre-OST0002_UUID -e lustre-OST0004_UUID -e lustre-OST0006_UUID  | sort -u | tr '\n' ' ' 
[10657.822000] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1 | grep lustre-OST0000_UUID
[10665.105651] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.cea1
[10669.770450] Lustre: DEBUG MARKER: lctl get_param -n 		lov.lustre-*.pools.cea1 2>/dev/null || echo foo
[10671.762637] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204a: Print default stripe attributes ===== 02:28:55 \(1734402535\)
[10672.129275] Lustre: DEBUG MARKER: == sanity test 204a: Print default stripe attributes ===== 02:28:55 (1734402535)
[10674.078422] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204b: Print default stripe size and offset ========================================================== 02:28:57 \(1734402537\)
[10674.445100] Lustre: DEBUG MARKER: == sanity test 204b: Print default stripe size and offset ========================================================== 02:28:57 (1734402537)
[10676.368607] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204c: Print default stripe count and offset ========================================================== 02:28:59 \(1734402539\)
[10676.733791] Lustre: DEBUG MARKER: == sanity test 204c: Print default stripe count and offset ========================================================== 02:28:59 (1734402539)
[10678.645278] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204d: Print default stripe count and size ========================================================== 02:29:02 \(1734402542\)
[10679.010416] Lustre: DEBUG MARKER: == sanity test 204d: Print default stripe count and size ========================================================== 02:29:02 (1734402542)
[10680.946805] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204e: Print raw stripe attributes ========= 02:29:04 \(1734402544\)
[10681.312212] Lustre: DEBUG MARKER: == sanity test 204e: Print raw stripe attributes ========= 02:29:04 (1734402544)
[10683.268158] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204f: Print raw stripe size and offset ==== 02:29:06 \(1734402546\)
[10683.635171] Lustre: DEBUG MARKER: == sanity test 204f: Print raw stripe size and offset ==== 02:29:06 (1734402546)
[10685.544448] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204g: Print raw stripe count and offset === 02:29:09 \(1734402549\)
[10685.917975] Lustre: DEBUG MARKER: == sanity test 204g: Print raw stripe count and offset === 02:29:09 (1734402549)
[10687.840104] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 204h: Print raw stripe count and size ===== 02:29:11 \(1734402551\)
[10688.208112] Lustre: DEBUG MARKER: == sanity test 204h: Print raw stripe count and size ===== 02:29:11 (1734402551)
[10690.125649] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205a: Verify job stats ==================== 02:29:13 \(1734402553\)
[10690.490024] Lustre: DEBUG MARKER: == sanity test 205a: Verify job stats ==================== 02:29:13 (1734402553)
[10690.610055] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10690.626653] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10690.962403] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10691.978827] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10692.995697] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10694.990108] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/bin\/lfs mkdir -i 0 -c 1 \/mnt\/lustre\/d205a.sanity
[10695.357921] Lustre: DEBUG MARKER: Test: /usr/bin/lfs mkdir -i 0 -c 1 /mnt/lustre/d205a.sanity
[10695.780524] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.lfs.15370
[10696.145958] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.15370
[10696.894406] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: rmdir \/mnt\/lustre\/d205a.sanity
[10697.257088] Lustre: DEBUG MARKER: Test: rmdir /mnt/lustre/d205a.sanity
[10697.686100] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.rmdir.18246
[10698.050064] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.rmdir.18246
[10698.788651] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: mknod \/mnt\/lustre\/f205a.sanity c 1 3
[10699.172383] Lustre: DEBUG MARKER: Test: mknod /mnt/lustre/f205a.sanity c 1 3
[10699.601387] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.mknod.3000
[10699.972981] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.mknod.3000
[10700.730518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: rm -f \/mnt\/lustre\/f205a.sanity
[10701.097964] Lustre: DEBUG MARKER: Test: rm -f /mnt/lustre/f205a.sanity
[10701.528698] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.rm.32055
[10701.896986] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.rm.32055
[10702.641263] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/bin\/lfs setstripe -i 0 -c 1 \/mnt\/lustre\/f205a.sanity
[10703.005720] Lustre: DEBUG MARKER: Test: /usr/bin/lfs setstripe -i 0 -c 1 /mnt/lustre/f205a.sanity
[10703.445302] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.lfs.9843
[10703.813053] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.9843
[10704.572957] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[10704.938972] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[10705.364430] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.touch.1622
[10705.729815] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.touch.1622
[10707.867341] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: dd if=\/dev\/zero of=\/mnt\/lustre\/f205a.sanity bs=1M count=1 oflag=sync
[10708.235635] Lustre: DEBUG MARKER: Test: dd if=/dev/zero of=/mnt/lustre/f205a.sanity bs=1M count=1 oflag=sync
[10708.662714] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.dd.21635
[10709.024859] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.dd.21635
[10709.907927] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: dd if=\/mnt\/lustre\/f205a.sanity of=\/dev\/null bs=1M count=1 iflag=direct
[10710.272830] Lustre: DEBUG MARKER: Test: dd if=/mnt/lustre/f205a.sanity of=/dev/null bs=1M count=1 iflag=direct
[10710.700713] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.dd.1444
[10711.069902] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.dd.1444
[10711.830582] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/lib64\/lustre\/tests\/truncate \/mnt\/lustre\/f205a.sanity 0
[10712.195759] Lustre: DEBUG MARKER: Test: /usr/lib64/lustre/tests/truncate /mnt/lustre/f205a.sanity 0
[10712.619360] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.truncate.10759
[10712.990048] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.truncate.10759
[10713.506519] Autotest: Test running for 175 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[10714.085282] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: mv -f \/mnt\/lustre\/f205a.sanity \/mnt\/lustre\/d205a.sanity.rename
[10714.454121] Lustre: DEBUG MARKER: Test: mv -f /mnt/lustre/f205a.sanity /mnt/lustre/d205a.sanity.rename
[10714.876527] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.mv.1006
[10715.239493] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.mv.1006
[10715.981866] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: \/usr\/bin\/lfs mkdir -i 0 -c 1 \/mnt\/lustre\/d205a.sanity.expire
[10716.343370] Lustre: DEBUG MARKER: Test: /usr/bin/lfs mkdir -i 0 -c 1 /mnt/lustre/d205a.sanity.expire
[10716.773963] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment nodelocal=id.205a.lfs.9253
[10717.143187] Lustre: DEBUG MARKER: Using JobID environment nodelocal=id.205a.lfs.9253
[10717.822433] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10717.839667] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10718.174684] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10719.191261] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10720.208154] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10720.695289] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[10721.061797] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[10721.485166] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment USER=S.root.touch.0.onyx-138vm1.onyx
[10721.853454] Lustre: DEBUG MARKER: Using JobID environment USER=S.root.touch.0.onyx-138vm1.onyx
[10722.615419] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[10722.986569] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[10723.421261] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment USER=S.root.touch.0.onyx-138vm1.E
[10723.791388] Lustre: DEBUG MARKER: Using JobID environment USER=S.root.touch.0.onyx-138vm1.E
[10724.557681] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Test: touch \/mnt\/lustre\/f205a.sanity
[10724.927861] Lustre: DEBUG MARKER: Test: touch /mnt/lustre/f205a.sanity
[10725.357414] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using JobID environment session=S.root.touch.0.onyx-138vm1.onyx
[10725.727421] Lustre: DEBUG MARKER: Using JobID environment session=S.root.touch.0.onyx-138vm1.onyx
[10728.754232] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10728.771903] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10729.107054] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10730.124297] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10731.141677] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[10731.593724] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205b: Verify job stats jobid and output format ========================================================== 02:29:55 \(1734402595\)
[10731.960538] Lustre: DEBUG MARKER: == sanity test 205b: Verify job stats jobid and output format ========================================================== 02:29:55 (1734402595)
[10735.748643] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205c: Verify client stats format ========== 02:29:59 \(1734402599\)
[10736.114516] Lustre: DEBUG MARKER: == sanity test 205c: Verify client stats format ========== 02:29:59 (1734402599)
[10737.977006] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205d: verify the format of some stats files ========================================================== 02:30:01 \(1734402601\)
[10738.338957] Lustre: DEBUG MARKER: == sanity test 205d: verify the format of some stats files ========================================================== 02:30:01 (1734402601)
[10742.069914] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205e: verify the output of lljobstat ====== 02:30:05 \(1734402605\)
[10742.432963] Lustre: DEBUG MARKER: == sanity test 205e: verify the output of lljobstat ====== 02:30:05 (1734402605)
[10748.084155] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205f: verify qos_ost_weights YAML format == 02:30:11 \(1734402611\)
[10748.450580] Lustre: DEBUG MARKER: == sanity test 205f: verify qos_ost_weights YAML format == 02:30:11 (1734402611)
[10750.979288] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205g: stress test for job_stats procfile == 02:30:14 \(1734402614\)
[10751.352791] Lustre: DEBUG MARKER: == sanity test 205g: stress test for job_stats procfile == 02:30:14 (1734402614)
[10845.220052] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205h: check jobid xattr is stored correctly ========================================================== 02:31:48 \(1734402708\)
[10845.590165] Lustre: DEBUG MARKER: == sanity test 205h: check jobid xattr is stored correctly ========================================================== 02:31:48 (1734402708)
[10850.247022] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205i: check job_xattr parameter accepts and rejects values correctly ========================================================== 02:31:53 \(1734402713\)
[10850.618087] Lustre: DEBUG MARKER: == sanity test 205i: check job_xattr parameter accepts and rejects values correctly ========================================================== 02:31:53 (1734402713)
[10854.946917] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 205k: Verify \'?\' operator on job stats ==== 02:31:58 \(1734402718\)
[10855.312136] Lustre: DEBUG MARKER: == sanity test 205k: Verify '?' operator on job stats ==== 02:31:58 (1734402718)
[10858.644542] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 206: fail lov_init_raid0\(\) doesn\'t lbug === 02:32:02 \(1734402722\)
[10859.014295] Lustre: DEBUG MARKER: == sanity test 206: fail lov_init_raid0() doesn't lbug === 02:32:02 (1734402722)
[10859.044861] Lustre: *** cfs_fail_loc=1403, val=1***
[10860.940048] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 207a: can refresh layout at glimpse ======= 02:32:04 \(1734402724\)
[10861.312746] Lustre: DEBUG MARKER: == sanity test 207a: can refresh layout at glimpse ======= 02:32:04 (1734402724)
[10863.250039] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 207b: can refresh layout at open ========== 02:32:06 \(1734402726\)
[10863.617872] Lustre: DEBUG MARKER: == sanity test 207b: can refresh layout at open ========== 02:32:06 (1734402726)
[10865.568010] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 208: Exclusive open ======================= 02:32:09 \(1734402729\)
[10865.938686] Lustre: DEBUG MARKER: == sanity test 208: Exclusive open ======================= 02:32:09 (1734402729)
[10887.017490] Lustre: lustre-MDT0000-mdc-ffff988c0b80b000: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[10887.020700] Lustre: Skipped 1 previous similar message
[10887.022092] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[10887.028412] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f134ff66 to 0xdf05ec28f148dbc0
[10887.030944] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[10887.032729] Lustre: Skipped 1 previous similar message
[10887.049266] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988c5f9d89c0 x1818651059799552/t42949740341(42949740341) o101->lustre-MDT0000-mdc-ffff988c0b80b000@10.240.22.189@tcp:12/10 lens 576/608 e 0 to 0 dl 1734402766 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'multiop.0' uid:0 gid:0
[10890.501409] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[10891.767751] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[10892.271573] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[10892.443191] Lustre: DEBUG MARKER: lctl get_param -n at_max
[10892.849249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[10892.936040] Lustre: 432371:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734402740/real 1734402740]  req@ffff988c5f9daa40 x1818651059801344/t0(0) o400->lustre-MDT0000-mdc-ffff988c0b80b000@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734402756 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[10893.256214] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[10897.256332] Lustre: 432372:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734402745/real 1734402745]  req@ffff988c5f9da080 x1818651059802496/t0(0) o400->lustre-MDT0000-mdc-ffff988c0b80b000@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734402761 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[10912.617741] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[10912.621074] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f148dbc0 to 0xdf05ec28f148df0f
[10912.625538] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988c109689c0 x1818651059811456/t47244640259(47244640259) o101->lustre-MDT0000-mdc-ffff988c0b80b000@10.240.22.189@tcp:12/10 lens 584/608 e 0 to 0 dl 1734402792 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'multiop.0' uid:0 gid:0
[10912.630731] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 1 previous similar message
[10915.309373] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[10916.578418] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[10917.070613] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[10917.170383] Lustre: DEBUG MARKER: lctl get_param -n at_max
[10917.567240] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[10917.994109] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[10918.759881] Lustre: 432372:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734402766/real 1734402766]  req@ffff988c5f9d96c0 x1818651059813504/t0(0) o400->lustre-MDT0000-mdc-ffff988c0b80b000@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734402782 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[10921.628364] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 209: read-only open\/close requests should be freed promptly ========================================================== 02:33:05 \(1734402785\)
[10921.996117] Lustre: DEBUG MARKER: == sanity test 209: read-only open/close requests should be freed promptly ========================================================== 02:33:05 (1734402785)
[10927.791023] bash (608016): drop_caches: 3
[10928.995500] bash (608016): drop_caches: 3
[10930.920997] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 210: lfs getstripe does not break leases == 02:33:14 \(1734402794\)
[10931.289678] Lustre: DEBUG MARKER: == sanity test 210: lfs getstripe does not break leases == 02:33:14 (1734402794)
[10935.190430] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 211: failed mirror split doesn\'t break write lease ========================================================== 02:33:18 \(1734402798\)
[10935.550628] Lustre: DEBUG MARKER: == sanity test 211: failed mirror split doesn't break write lease ========================================================== 02:33:18 (1734402798)
[10938.279725] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 212: Sendfile test ====================================================================================================== 02:33:21 \(1734402801\)
[10938.646163] Lustre: DEBUG MARKER: == sanity test 212: Sendfile test ====================================================================================================== 02:33:21 (1734402801)
[10940.696286] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 213: OSC lock completion and cancel race don\'t crash - bug 18829 ========================================================== 02:33:24 \(1734402804\)
[10941.063683] Lustre: DEBUG MARKER: == sanity test 213: OSC lock completion and cancel race don't crash - bug 18829 ========================================================== 02:33:24 (1734402804)
[10941.118851] LustreError: 432371:0:(osc_request.c:3109:osc_enqueue_interpret()) cfs_fail_timeout id 40f sleeping for 10000ms
[10951.207678] LustreError: 432371:0:(osc_request.c:3109:osc_enqueue_interpret()) cfs_fail_timeout id 40f awake
[10953.139397] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 214: hash-indexed directory test - bug 20133 ========================================================== 02:33:36 \(1734402816\)
[10953.503035] Lustre: DEBUG MARKER: == sanity test 214: hash-indexed directory test - bug 20133 ========================================================== 02:33:36 (1734402816)
[10958.638003] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 215: lnet exists and has proper content - bugs 18102, 21079, 21517 ========================================================== 02:33:42 \(1734402822\)
[10959.010366] Lustre: DEBUG MARKER: == sanity test 215: lnet exists and has proper content - bugs 18102, 21079, 21517 ========================================================== 02:33:42 (1734402822)
[10961.021187] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 216: check lockless direct write updates file size and kms correctly ========================================================== 02:33:44 \(1734402824\)
[10961.392360] Lustre: DEBUG MARKER: == sanity test 216: check lockless direct write updates file size and kms correctly ========================================================== 02:33:44 (1734402824)
[10961.489408] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param osc.*.contention_seconds);
[10961.489408] 			 [[ -z "" ]] && param= ||
[10961.489408] 			 param=$(grep  <<< "$params");
[10961.489408] 			 [[ -z $param ]] && param="$params";
[10961.489408] 			 while read s; do echo client $s;
[10961.489408] 			 done <<< "$param"
[10969.466475] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n =
[10977.948034] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 217: check lctl ping for hostnames with embedded hyphen \(\'-\'\) ========================================================== 02:34:01 \(1734402841\)
[10978.316104] Lustre: DEBUG MARKER: == sanity test 217: check lctl ping for hostnames with embedded hyphen ('-') ========================================================== 02:34:01 (1734402841)
[10978.406018] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[10978.423057] Lustre: DEBUG MARKER: getent ahostsv4 onyx-138vm1.onyx.whamcloud.com
[10982.921117] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 218: parallel read and truncate should not deadlock ========================================================== 02:34:06 \(1734402846\)
[10983.285736] Lustre: DEBUG MARKER: == sanity test 218: parallel read and truncate should not deadlock ========================================================== 02:34:06 (1734402846)
[10983.708778] Lustre: DEBUG MARKER: /usr/sbin/lctl mark creating a 10 Mb file
[10984.070349] Lustre: DEBUG MARKER: creating a 10 Mb file
[10996.079377] Lustre: DEBUG MARKER: /usr/sbin/lctl mark starting reads
[10996.526941] Lustre: DEBUG MARKER: starting reads
[10997.012009] Lustre: DEBUG MARKER: /usr/sbin/lctl mark truncating the file
[10997.376098] Lustre: DEBUG MARKER: truncating the file
[10997.883556] Lustre: DEBUG MARKER: /usr/sbin/lctl mark killing dd
[10998.245996] Lustre: DEBUG MARKER: killing dd
[10998.664652] Lustre: DEBUG MARKER: /usr/sbin/lctl mark removing the temporary file
[10999.030730] Lustre: DEBUG MARKER: removing the temporary file
[11000.909280] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 219: LU-394: Write partial won\'t cause uncontiguous pages vec at LND ========================================================== 02:34:24 \(1734402864\)
[11001.276861] Lustre: DEBUG MARKER: == sanity test 219: LU-394: Write partial won't cause uncontiguous pages vec at LND ========================================================== 02:34:24 (1734402864)
[11001.304266] Lustre: *** cfs_fail_loc=411, val=0***
[11003.322137] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 220: preallocated MDS objects still used if ENOSPC from OST ========================================================== 02:34:26 \(1734402866\)
[11003.687232] Lustre: DEBUG MARKER: == sanity test 220: preallocated MDS objects still used if ENOSPC from OST ========================================================== 02:34:26 (1734402866)
[11007.381032] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_220 		2>/dev/null || echo foo
[11014.435917] Autotest: Test running for 180 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[11018.055447] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 221: make sure fault and truncate race to not cause OOM ========================================================== 02:34:41 \(1734402881\)
[11018.423244] Lustre: DEBUG MARKER: == sanity test 221: make sure fault and truncate race to not cause OOM ========================================================== 02:34:41 (1734402881)
[11026.773221] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 222a: AGL for ls should not trigger CLIO lock failure ========================================================== 02:34:50 \(1734402890\)
[11027.144796] Lustre: DEBUG MARKER: == sanity test 222a: AGL for ls should not trigger CLIO lock failure ========================================================== 02:34:50 (1734402890)
[11029.135992] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 222b: AGL for rmdir should not trigger CLIO lock failure ========================================================== 02:34:52 \(1734402892\)
[11029.506259] Lustre: DEBUG MARKER: == sanity test 222b: AGL for rmdir should not trigger CLIO lock failure ========================================================== 02:34:52 (1734402892)
[11031.505864] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 223: osc reenqueue if without AGL lock granted ================================================================================= 02:34:54 \(1734402894\)
[11031.873447] Lustre: DEBUG MARKER: == sanity test 223: osc reenqueue if without AGL lock granted ================================================================================= 02:34:54 (1734402894)
[11033.873699] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224a: Don\'t panic on bulk IO failure ====== 02:34:57 \(1734402897\)
[11034.246112] Lustre: DEBUG MARKER: == sanity test 224a: Don't panic on bulk IO failure ====== 02:34:57 (1734402897)
[11034.294418] Lustre: *** cfs_fail_loc=508, val=2147483648***
[11034.295554] LustreError: 432363:0:(events.c:209:client_bulk_callback()) event type 1, status -5, desc ffff988c108e4000
[11034.297476] Lustre: 432369:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1734402898/real 1734402898]  req@ffff988d08e05380 x1818651060511872/t0(0) o4->lustre-OST0005-osc-ffff988c0b80b000@10.240.22.182@tcp:6/4 lens 488/448 e 0 to 1 dl 1734402914 ref 2 fl Rpc:eXQr/200/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0
[11034.302907] Lustre: 432369:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[11034.304715] Lustre: lustre-OST0005-osc-ffff988c0b80b000: Connection to lustre-OST0005 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[11034.307752] Lustre: Skipped 1 previous similar message
[11038.308629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224b: Don\'t panic on bulk IO failure ====== 02:35:01 \(1734402901\)
[11038.688336] Lustre: DEBUG MARKER: == sanity test 224b: Don't panic on bulk IO failure ====== 02:35:01 (1734402901)
[11039.859719] Lustre: DEBUG MARKER: lctl set_param at_max=0
[11045.770694] Lustre: DEBUG MARKER: lctl set_param at_max=600
[11046.584526] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224c: Don\'t hang if one of md lost during large bulk RPC ========================================================== 02:35:10 \(1734402910\)
[11046.953346] Lustre: DEBUG MARKER: == sanity test 224c: Don't hang if one of md lost during large bulk RPC ========================================================== 02:35:10 (1734402910)
[11049.716992] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11049.733545] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11050.071583] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11051.088778] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11052.105850] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11052.123196] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11052.139577] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11052.475369] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11053.492721] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11059.047061] Lustre: 432371:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734402917/real 1734402917]  req@ffff988c1096a3c0 x1818651060526080/t0(0) o4->lustre-OST0000-osc-ffff988c0b80b000@10.240.22.182@tcp:6/4 lens 488/448 e 0 to 1 dl 1734402922 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0
[11059.519471] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11059.536228] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11059.874336] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11060.891203] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11061.908172] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_max
[11061.925489] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11061.942081] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11062.294253] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11063.311339] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n timeout
[11067.525992] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 224d: Don\'t corrupt data on bulk IO timeout ========================================================== 02:35:31 \(1734402931\)
[11067.894008] Lustre: DEBUG MARKER: == sanity test 224d: Don't corrupt data on bulk IO timeout ========================================================== 02:35:31 (1734402931)
[11068.991261] Lustre: DEBUG MARKER: lctl set_param at_max=0
[11091.046948] LustreError: 432370:0:(client.c:2233:ptlrpc_check_set()) @@@ bulk transfer failed 0/1048576/0  req@ffff988d08e06d80 x1818651060537472/t0(0) o3->lustre-OST0000-osc-ffff988c0b80b000@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 1 dl 1734402953 ref 2 fl Bulk:ReXMQU/200/0 rc 0/0 job:'dd.0' uid:0 gid:0
[11091.051769] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d08e06d80 x1818651060537472/t0(0) o3->lustre-OST0000-osc-ffff988c0b80b000@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 1 dl 1734402953 ref 2 fl Interpret:ReXMQU/200/0 rc -5/0 job:'dd.0' uid:0 gid:0
[11091.059614] Lustre: lustre-OST0000-osc-ffff988c0b80b000: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[11091.061636] Lustre: Skipped 3 previous similar messages
[11094.387247] Lustre: DEBUG MARKER: lctl set_param at_max=600
[11095.195455] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 225a: Metadata survey sanity with zero-stripe ========================================================== 02:35:58 \(1734402958\)
[11095.562330] Lustre: DEBUG MARKER: == sanity test 225a: Metadata survey sanity with zero-stripe ========================================================== 02:35:58 (1734402958)
[11101.224738] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 225b: Metadata survey sanity with stripe_count = 1 ========================================================== 02:36:04 \(1734402964\)
[11101.599524] Lustre: DEBUG MARKER: == sanity test 225b: Metadata survey sanity with stripe_count = 1 ========================================================== 02:36:04 (1734402964)
[11107.415460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226a: call path2fid and fid2path on files of all type ========================================================== 02:36:10 \(1734402970\)
[11107.781576] Lustre: DEBUG MARKER: == sanity test 226a: call path2fid and fid2path on files of all type ========================================================== 02:36:10 (1734402970)
[11109.756199] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226b: call path2fid and fid2path on files of all type under remote dir ========================================================== 02:36:13 \(1734402973\)
[11110.130303] Lustre: DEBUG MARKER: == sanity test 226b: call path2fid and fid2path on files of all type under remote dir ========================================================== 02:36:13 (1734402973)
[11110.553276] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_226b needs \>= 2 MDTs
[11110.918225] Lustre: DEBUG MARKER: SKIP: sanity test_226b needs >= 2 MDTs
[11111.372926] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226c: call path2fid and fid2path under remote dir with subdir mount ========================================================== 02:36:14 \(1734402974\)
[11111.744050] Lustre: DEBUG MARKER: == sanity test 226c: call path2fid and fid2path under remote dir with subdir mount ========================================================== 02:36:14 (1734402974)
[11112.168593] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_226c needs \>= 2 MDTs
[11112.535263] Lustre: DEBUG MARKER: SKIP: sanity test_226c needs >= 2 MDTs
[11112.991801] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226d: verify fid2path with -n and -fn option ========================================================== 02:36:16 \(1734402976\)
[11113.357079] Lustre: DEBUG MARKER: == sanity test 226d: verify fid2path with -n and -fn option ========================================================== 02:36:16 (1734402976)
[11115.479712] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 226e: Verify path2fid -0 option with newline and space ========================================================== 02:36:18 \(1734402978\)
[11115.847782] Lustre: DEBUG MARKER: == sanity test 226e: Verify path2fid -0 option with newline and space ========================================================== 02:36:18 (1734402978)
[11117.937870] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 227: running truncated executable does not cause OOM ========================================================== 02:36:21 \(1734402981\)
[11118.304299] Lustre: DEBUG MARKER: == sanity test 227: running truncated executable does not cause OOM ========================================================== 02:36:21 (1734402981)
[11120.273834] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 228a: try to reuse idle OI blocks ========= 02:36:23 \(1734402983\)
[11120.641029] Lustre: DEBUG MARKER: == sanity test 228a: try to reuse idle OI blocks ========= 02:36:23 (1734402983)
[11121.364814] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11121.740114] Lustre: *** cfs_fail_loc=1002, val=0***
[11126.896815] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11135.203247] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11138.341186] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11139.776722] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11141.460336] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11151.812052] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 228b: idle OI blocks can be reused after MDT restart ========================================================== 02:36:55 \(1734403015\)
[11152.184210] Lustre: DEBUG MARKER: == sanity test 228b: idle OI blocks can be reused after MDT restart ========================================================== 02:36:55 (1734403015)
[11152.922330] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11153.303455] Lustre: *** cfs_fail_loc=1002, val=0***
[11158.477147] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11166.883673] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11170.164262] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11178.087641] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[11178.092395] Lustre: lustre-MDT0000-mdc-ffff988c0b80b000: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[11178.095496] Lustre: Skipped 2 previous similar messages
[11178.096668] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f148df0f to 0xdf05ec28f15debc6
[11180.565243] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11182.227456] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11191.603224] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 228c: NOT shrink the last entry in OI index node to recycle idle leaf ========================================================== 02:37:35 \(1734403055\)
[11191.973254] Lustre: DEBUG MARKER: == sanity test 228c: NOT shrink the last entry in OI index node to recycle idle leaf ========================================================== 02:37:35 (1734403055)
[11192.694404] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11193.073283] Lustre: *** cfs_fail_loc=1002, val=0***
[11202.610262] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11210.867619] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11216.520537] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11217.972105] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11219.650542] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11219.942150] Lustre: 432372:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734403036/real 1734403036]  req@ffff988d02015a00 x1818651076955136/t0(0) o400->lustre-MDT0000-mdc-ffff988c0b80b000@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734403083 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[11219.947872] Lustre: 432372:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[11229.167148] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 229: getstripe\/stat\/rm\/attr changes work on released files ========================================================== 02:38:12 \(1734403092\)
[11229.538134] Lustre: DEBUG MARKER: == sanity test 229: getstripe/stat/rm/attr changes work on released files ========================================================== 02:38:12 (1734403092)
[11231.488079] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230a: Create remote directory and files under the remote directory ========================================================== 02:38:14 \(1734403094\)
[11231.854243] Lustre: DEBUG MARKER: == sanity test 230a: Create remote directory and files under the remote directory ========================================================== 02:38:14 (1734403094)
[11232.278644] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230a needs \>= 2 MDTs
[11232.644086] Lustre: DEBUG MARKER: SKIP: sanity test_230a needs >= 2 MDTs
[11233.098726] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230b: migrate directory =================== 02:38:16 \(1734403096\)
[11233.467905] Lustre: DEBUG MARKER: == sanity test 230b: migrate directory =================== 02:38:16 (1734403096)
[11233.889112] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230b needs \>= 2 MDTs
[11234.258114] Lustre: DEBUG MARKER: SKIP: sanity test_230b needs >= 2 MDTs
[11234.715862] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230c: check directory accessiblity if migration failed ========================================================== 02:38:18 \(1734403098\)
[11235.087055] Lustre: DEBUG MARKER: == sanity test 230c: check directory accessiblity if migration failed ========================================================== 02:38:18 (1734403098)
[11235.507465] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230c needs \>= 2 MDTs
[11235.875044] Lustre: DEBUG MARKER: SKIP: sanity test_230c needs >= 2 MDTs
[11236.312570] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230d skipping SLOW test 230d
[11236.680853] Lustre: DEBUG MARKER: SKIP: sanity test_230d skipping SLOW test 230d
[11237.130136] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230e: migrate mulitple local link files === 02:38:20 \(1734403100\)
[11237.499307] Lustre: DEBUG MARKER: == sanity test 230e: migrate mulitple local link files === 02:38:20 (1734403100)
[11237.921622] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230e needs \>= 2 MDTs
[11238.290064] Lustre: DEBUG MARKER: SKIP: sanity test_230e needs >= 2 MDTs
[11238.747256] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230f: migrate mulitple remote link files == 02:38:22 \(1734403102\)
[11239.116056] Lustre: DEBUG MARKER: == sanity test 230f: migrate mulitple remote link files == 02:38:22 (1734403102)
[11239.542073] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230f needs \>= 2 MDTs
[11239.907480] Lustre: DEBUG MARKER: SKIP: sanity test_230f needs >= 2 MDTs
[11240.361004] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230g: migrate dir to non-exist MDT ======== 02:38:23 \(1734403103\)
[11240.727873] Lustre: DEBUG MARKER: == sanity test 230g: migrate dir to non-exist MDT ======== 02:38:23 (1734403103)
[11241.147106] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230g needs \>= 2 MDTs
[11241.522730] Lustre: DEBUG MARKER: SKIP: sanity test_230g needs >= 2 MDTs
[11241.987326] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230h: migrate .. and root ================= 02:38:25 \(1734403105\)
[11242.352524] Lustre: DEBUG MARKER: == sanity test 230h: migrate .. and root ================= 02:38:25 (1734403105)
[11242.772023] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230h needs \>= 2 MDTs
[11243.137384] Lustre: DEBUG MARKER: SKIP: sanity test_230h needs >= 2 MDTs
[11243.595425] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230i: lfs migrate -m tolerates trailing slashes ========================================================== 02:38:27 \(1734403107\)
[11243.963876] Lustre: DEBUG MARKER: == sanity test 230i: lfs migrate -m tolerates trailing slashes ========================================================== 02:38:27 (1734403107)
[11244.389139] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230i needs \>= 2 MDTs
[11244.755625] Lustre: DEBUG MARKER: SKIP: sanity test_230i needs >= 2 MDTs
[11245.210434] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230j: DoM file data not changed after dir migration ========================================================== 02:38:28 \(1734403108\)
[11245.579214] Lustre: DEBUG MARKER: == sanity test 230j: DoM file data not changed after dir migration ========================================================== 02:38:28 (1734403108)
[11245.999242] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230j needs \>= 2 MDTs
[11246.363623] Lustre: DEBUG MARKER: SKIP: sanity test_230j needs >= 2 MDTs
[11246.813744] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230k: file data not changed after dir migration ========================================================== 02:38:30 \(1734403110\)
[11247.177898] Lustre: DEBUG MARKER: == sanity test 230k: file data not changed after dir migration ========================================================== 02:38:30 (1734403110)
[11247.600793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230k needs \>= 4 MDTs
[11247.966039] Lustre: DEBUG MARKER: SKIP: sanity test_230k needs >= 4 MDTs
[11248.465835] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230l: readdir between MDTs won\'t crash ==== 02:38:31 \(1734403111\)
[11248.827967] Lustre: DEBUG MARKER: == sanity test 230l: readdir between MDTs won't crash ==== 02:38:31 (1734403111)
[11249.246224] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230l needs \>= 2 MDTs
[11249.612075] Lustre: DEBUG MARKER: SKIP: sanity test_230l needs >= 2 MDTs
[11250.067270] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230m: xattrs not changed after dir migration ========================================================== 02:38:33 \(1734403113\)
[11250.430668] Lustre: DEBUG MARKER: == sanity test 230m: xattrs not changed after dir migration ========================================================== 02:38:33 (1734403113)
[11250.851734] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230m needs \>= 2 MDTs
[11251.215228] Lustre: DEBUG MARKER: SKIP: sanity test_230m needs >= 2 MDTs
[11251.663249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230n: Dir migration with mirrored file ==== 02:38:35 \(1734403115\)
[11252.024487] Lustre: DEBUG MARKER: == sanity test 230n: Dir migration with mirrored file ==== 02:38:35 (1734403115)
[11252.445753] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230n needs \>= 2 MDTs
[11252.810165] Lustre: DEBUG MARKER: SKIP: sanity test_230n needs >= 2 MDTs
[11253.265991] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230o: dir split =========================== 02:38:36 \(1734403116\)
[11253.637909] Lustre: DEBUG MARKER: == sanity test 230o: dir split =========================== 02:38:36 (1734403116)
[11254.059495] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230o needs \>= 2 MDTs
[11254.424529] Lustre: DEBUG MARKER: SKIP: sanity test_230o needs >= 2 MDTs
[11254.882492] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230p: dir merge =========================== 02:38:38 \(1734403118\)
[11255.250804] Lustre: DEBUG MARKER: == sanity test 230p: dir merge =========================== 02:38:38 (1734403118)
[11255.670993] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230p needs \>= 2 MDTs
[11256.035092] Lustre: DEBUG MARKER: SKIP: sanity test_230p needs >= 2 MDTs
[11256.489040] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230q: dir auto split ====================== 02:38:39 \(1734403119\)
[11256.854749] Lustre: DEBUG MARKER: == sanity test 230q: dir auto split ====================== 02:38:39 (1734403119)
[11257.275192] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230q needs \>= 2 MDTs
[11257.638144] Lustre: DEBUG MARKER: SKIP: sanity test_230q needs >= 2 MDTs
[11258.091901] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230r: migrate with too many local locks === 02:38:41 \(1734403121\)
[11258.456514] Lustre: DEBUG MARKER: == sanity test 230r: migrate with too many local locks === 02:38:41 (1734403121)
[11258.872089] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230r needs \>= 2 MDTs
[11259.240524] Lustre: DEBUG MARKER: SKIP: sanity test_230r needs >= 2 MDTs
[11259.701629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230s: lfs mkdir should return -EEXIST if target exists ========================================================== 02:38:43 \(1734403123\)
[11260.069775] Lustre: DEBUG MARKER: == sanity test 230s: lfs mkdir should return -EEXIST if target exists ========================================================== 02:38:43 (1734403123)
[11263.405249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230t: migrate directory with project ID set ========================================================== 02:38:46 \(1734403126\)
[11263.770862] Lustre: DEBUG MARKER: == sanity test 230t: migrate directory with project ID set ========================================================== 02:38:46 (1734403126)
[11264.192944] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230t needs \>= 2 MDTs
[11264.556329] Lustre: DEBUG MARKER: SKIP: sanity test_230t needs >= 2 MDTs
[11265.007185] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230u: migrate directory by QOS ============ 02:38:48 \(1734403128\)
[11265.368670] Lustre: DEBUG MARKER: == sanity test 230u: migrate directory by QOS ============ 02:38:48 (1734403128)
[11265.800219] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230u needs \>= 4 MDTs
[11266.163047] Lustre: DEBUG MARKER: SKIP: sanity test_230u needs >= 4 MDTs
[11266.614521] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230v: subdir migrated to the MDT where its parent is located ========================================================== 02:38:50 \(1734403130\)
[11266.979184] Lustre: DEBUG MARKER: == sanity test 230v: subdir migrated to the MDT where its parent is located ========================================================== 02:38:50 (1734403130)
[11267.399064] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230v needs \>= 4 MDTs
[11267.763203] Lustre: DEBUG MARKER: SKIP: sanity test_230v needs >= 4 MDTs
[11268.219198] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230w: non-recursive mode dir migration ==== 02:38:51 \(1734403131\)
[11268.584730] Lustre: DEBUG MARKER: == sanity test 230w: non-recursive mode dir migration ==== 02:38:51 (1734403131)
[11269.002821] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230w needs \>= 2 MDTs
[11269.367648] Lustre: DEBUG MARKER: SKIP: sanity test_230w needs >= 2 MDTs
[11269.820132] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230x: dir migration check space =========== 02:38:53 \(1734403133\)
[11270.184825] Lustre: DEBUG MARKER: == sanity test 230x: dir migration check space =========== 02:38:53 (1734403133)
[11270.604065] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230x needs \>= 2 MDTs
[11270.962516] Lustre: DEBUG MARKER: SKIP: sanity test_230x needs >= 2 MDTs
[11271.417989] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230y: unlink dir with bad hash type ======= 02:38:54 \(1734403134\)
[11271.782709] Lustre: DEBUG MARKER: == sanity test 230y: unlink dir with bad hash type ======= 02:38:54 (1734403134)
[11272.204157] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230y needs \>= 2 MDTs
[11272.571234] Lustre: DEBUG MARKER: SKIP: sanity test_230y needs >= 2 MDTs
[11273.024299] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 230z: resume dir migration with bad hash type ========================================================== 02:38:56 \(1734403136\)
[11273.387748] Lustre: DEBUG MARKER: == sanity test 230z: resume dir migration with bad hash type ========================================================== 02:38:56 (1734403136)
[11273.807575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_230z needs \>= 2 MDTs
[11274.171797] Lustre: DEBUG MARKER: SKIP: sanity test_230z needs >= 2 MDTs
[11274.623060] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 231a: checking that reading\/writing of BRW RPC size results in one RPC ========================================================== 02:38:58 \(1734403138\)
[11274.987406] Lustre: DEBUG MARKER: == sanity test 231a: checking that reading/writing of BRW RPC size results in one RPC ========================================================== 02:38:58 (1734403138)
[11279.083546] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 231b: must not assert on fully utilized OST request buffer ========================================================== 02:39:02 \(1734403142\)
[11279.449664] Lustre: DEBUG MARKER: == sanity test 231b: must not assert on fully utilized OST request buffer ========================================================== 02:39:02 (1734403142)
[11288.971552] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 232a: failed lock should not block umount ========================================================== 02:39:12 \(1734403152\)
[11289.337218] Lustre: DEBUG MARKER: == sanity test 232a: failed lock should not block umount ========================================================== 02:39:12 (1734403152)
[11289.676909] LustreError: lustre-OST0000-osc-ffff988c0b80b000: operation ldlm_enqueue to node 10.240.22.182@tcp failed: rc = -12
[11289.991913] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[11290.004366] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[11290.185557] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[11291.410929] Lustre: Unmounted lustre-client
[11291.424422] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[11291.436713] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[11291.484832] Lustre: Mounted lustre-client
[11297.761715] Lustre: lustre-OST0000-osc-ffff988c08037800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[11297.763818] Lustre: Skipped 2 previous similar messages
[11301.773180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 232b: failed data version lock should not block umount ========================================================== 02:39:25 \(1734403165\)
[11302.135869] Lustre: DEBUG MARKER: == sanity test 232b: failed data version lock should not block umount ========================================================== 02:39:25 (1734403165)
[11303.486006] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[11303.498870] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[11303.668969] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[11303.713447] Lustre: Unmounted lustre-client
[11303.725184] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[11303.737936] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[11303.791846] Lustre: Mounted lustre-client
[11313.230816] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 233a: checking that OBF of the FS root succeeds ========================================================== 02:39:36 \(1734403176\)
[11313.600236] Lustre: DEBUG MARKER: == sanity test 233a: checking that OBF of the FS root succeeds ========================================================== 02:39:36 (1734403176)
[11315.250830] Autotest: Test running for 185 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[11315.616880] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 233b: checking that OBF of the FS .lustre succeeds ========================================================== 02:39:39 \(1734403179\)
[11315.982328] Lustre: DEBUG MARKER: == sanity test 233b: checking that OBF of the FS .lustre succeeds ========================================================== 02:39:39 (1734403179)
[11317.881175] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 234: xattr cache should not crash on ENOMEM ========================================================== 02:39:41 \(1734403181\)
[11318.250146] Lustre: DEBUG MARKER: == sanity test 234: xattr cache should not crash on ENOMEM ========================================================== 02:39:41 (1734403181)
[11318.265199] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[11318.265199] 			 [[ -z "" ]] && param= ||
[11318.265199] 			 param=$(grep  <<< "$params");
[11318.265199] 			 [[ -z $param ]] && param="$params";
[11318.265199] 			 while read s; do echo client $s;
[11318.265199] 			 done <<< "$param"
[11318.304350] Lustre: *** cfs_fail_loc=1405, val=0***
[11318.325399] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff988c2f055800.xattr_cache=1
[11320.197249] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 235: LU-1715: flock deadlock detection does not work properly ========================================================== 02:39:43 \(1734403183\)
[11320.564131] Lustre: DEBUG MARKER: == sanity test 235: LU-1715: flock deadlock detection does not work properly ========================================================== 02:39:43 (1734403183)
[11324.447468] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 236: Layout swap on open unlinked file ==== 02:39:47 \(1734403187\)
[11324.815679] Lustre: DEBUG MARKER: == sanity test 236: Layout swap on open unlinked file ==== 02:39:47 (1734403187)
[11326.877326] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 238: Verify linkea consistency ============ 02:39:50 \(1734403190\)
[11327.243747] Lustre: DEBUG MARKER: == sanity test 238: Verify linkea consistency ============ 02:39:50 (1734403190)
[11329.227562] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 239A: osp_sync test ======================= 02:39:52 \(1734403192\)
[11329.598009] Lustre: DEBUG MARKER: == sanity test 239A: osp_sync test ======================= 02:39:52 (1734403192)
[11330.326946] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11333.346188] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11334.785740] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[11336.652055] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[11340.871875] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 239a: process invalid osp sync record correctly ========================================================== 02:40:04 \(1734403204\)
[11341.238432] Lustre: DEBUG MARKER: == sanity test 239a: process invalid osp sync record correctly ========================================================== 02:40:04 (1734403204)
[11348.372008] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 239b: process osp sync record with ENOMEM error correctly ========================================================== 02:40:11 \(1734403211\)
[11348.739179] Lustre: DEBUG MARKER: == sanity test 239b: process osp sync record with ENOMEM error correctly ========================================================== 02:40:11 (1734403211)
[11355.078793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 240: race between ldlm enqueue and the connection RPC \(no ASSERT\) ========================================================== 02:40:18 \(1734403218\)
[11355.447186] Lustre: DEBUG MARKER: == sanity test 240: race between ldlm enqueue and the connection RPC (no ASSERT) ========================================================== 02:40:18 (1734403218)
[11355.865518] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_240 needs \>= 2 MDTs
[11356.229720] Lustre: DEBUG MARKER: SKIP: sanity test_240 needs >= 2 MDTs
[11356.686999] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 241a: bio vs dio ========================== 02:40:20 \(1734403220\)
[11357.053129] Lustre: DEBUG MARKER: == sanity test 241a: bio vs dio ========================== 02:40:20 (1734403220)
[11372.617524] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 241b: dio vs dio ========================== 02:40:36 \(1734403236\)
[11372.986068] Lustre: DEBUG MARKER: == sanity test 241b: dio vs dio ========================== 02:40:36 (1734403236)
[11378.986392] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 242: mdt_readpage failure should not cause directory unreadable ========================================================== 02:40:42 \(1734403242\)
[11379.352816] Lustre: DEBUG MARKER: == sanity test 242: mdt_readpage failure should not cause directory unreadable ========================================================== 02:40:42 (1734403242)
[11379.826937] LustreError: lustre-MDT0000-mdc-ffff988c2f055800: operation mds_readpage to node 10.240.22.189@tcp failed: rc = -12
[11381.984521] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 243: various group lock tests ============= 02:40:45 \(1734403245\)
[11382.348619] Lustre: DEBUG MARKER: == sanity test 243: various group lock tests ============= 02:40:45 (1734403245)
[11383.276649] Lustre: 656734:0:(file.c:2984:ll_get_grouplock()) lustre: group lock already existed with gid 97486: rc = -22
[11383.279067] Lustre: 656734:0:(file.c:3058:ll_put_grouplock()) lustre: no group lock held: rc = -22
[11383.280752] Lustre: 656734:0:(file.c:2967:ll_get_grouplock()) lustre: group id for group lock must not be 0: rc = -22
[11383.283706] Lustre: 656734:0:(file.c:3067:ll_put_grouplock()) lustre: group lock 4294967286 not match current id 3543: rc = -22
[11413.231516] Lustre: 656734:0:(file.c:3058:ll_put_grouplock()) lustre: no group lock held: rc = -22
[11413.240168] Lustre: 656734:0:(file.c:2967:ll_get_grouplock()) lustre: group id for group lock must not be 0: rc = -22
[11415.411074] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 244a: sendfile with group lock tests ====== 02:41:18 \(1734403278\)
[11415.778992] Lustre: DEBUG MARKER: == sanity test 244a: sendfile with group lock tests ====== 02:41:18 (1734403278)
[11434.868983] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 244b: multi-threaded write with group lock ========================================================== 02:41:38 \(1734403298\)
[11435.239517] Lustre: DEBUG MARKER: == sanity test 244b: multi-threaded write with group lock ========================================================== 02:41:38 (1734403298)
[11437.345420] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 245a: check mdc connection flag\/data: multiple modify RPCs ========================================================== 02:41:40 \(1734403300\)
[11437.714888] Lustre: DEBUG MARKER: == sanity test 245a: check mdc connection flag/data: multiple modify RPCs ========================================================== 02:41:40 (1734403300)
[11439.612655] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 245b: check osp connection flag\/data: multiple modify RPCs ========================================================== 02:41:43 \(1734403303\)
[11439.979343] Lustre: DEBUG MARKER: == sanity test 245b: check osp connection flag/data: multiple modify RPCs ========================================================== 02:41:43 (1734403303)
[11440.438984] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_245b needs \>= 2 MDTs
[11440.806831] Lustre: DEBUG MARKER: SKIP: sanity test_245b needs >= 2 MDTs
[11441.264689] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247a: mount subdir as fileset ============= 02:41:44 \(1734403304\)
[11441.628426] Lustre: DEBUG MARKER: == sanity test 247a: mount subdir as fileset ============= 02:41:44 (1734403304)
[11441.656338] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247a.sanity
[11441.668280] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre_d247a.sanity
[11441.721657] Lustre: Mounted lustre-client
[11441.733248] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[11441.747114] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247a.sanity//d247a.sanity
[11441.764134] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247a.sanity' ' /proc/mounts ||
[11441.764134] 			umount /mnt/lustre_d247a.sanity
[11441.812014] Lustre: Unmounted lustre-client
[11441.818512] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre/d247a.sanity /mnt/lustre_d247a.sanity
[11441.931850] Lustre: DEBUG MARKER: grep -c /mnt/lustre_d247a.sanity' ' /proc/mounts
[11441.944783] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_d247a.sanity
[11442.122297] Lustre: DEBUG MARKER: umount  /mnt/lustre_d247a.sanity 2>&1
[11444.032698] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247b: mount subdir that dose not exist ==== 02:41:47 \(1734403307\)
[11444.397896] Lustre: DEBUG MARKER: == sanity test 247b: mount subdir that dose not exist ==== 02:41:47 (1734403307)
[11444.422473] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247b.sanity
[11444.434518] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre/d247b.sanity /mnt/lustre_d247b.sanity
[11444.479671] LustreError: 659422:0:(llite_lib.c:710:client_common_fill_super()) cannot mds_connect: rc = -2
[11444.509502] Lustre: Unmounted lustre-client
[11444.510380] Lustre: Skipped 1 previous similar message
[11444.512662] LustreError: 659422:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -2
[11446.373496] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247c: running fid2path outside subdirectory root ========================================================== 02:41:49 \(1734403309\)
[11446.739463] Lustre: DEBUG MARKER: == sanity test 247c: running fid2path outside subdirectory root ========================================================== 02:41:49 (1734403309)
[11446.770561] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247c.sanity
[11446.782532] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre_d247c.sanity
[11446.828467] Lustre: Mounted lustre-client
[11446.831923] Lustre: Skipped 1 previous similar message
[11446.839896] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[11446.854758] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247c.sanity//d247c.sanity
[11446.871465] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247c.sanity' ' /proc/mounts ||
[11446.871465] 			umount /mnt/lustre_d247c.sanity
[11446.942728] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre/d247c.sanity /mnt/lustre_d247c.sanity
[11447.029716] Lustre: DEBUG MARKER: grep -c /mnt/lustre_d247c.sanity' ' /proc/mounts
[11447.042865] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_d247c.sanity
[11447.214270] Lustre: DEBUG MARKER: umount  /mnt/lustre_d247c.sanity 2>&1
[11449.125451] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247d: running fid2path inside subdirectory root ========================================================== 02:41:52 \(1734403312\)
[11449.493924] Lustre: DEBUG MARKER: == sanity test 247d: running fid2path inside subdirectory root ========================================================== 02:41:52 (1734403312)
[11449.525520] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247d.sanity
[11449.537491] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre_d247d.sanity
[11449.589550] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[11449.603333] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247d.sanity//d247d.sanity
[11449.620177] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247d.sanity' ' /proc/mounts ||
[11449.620177] 			umount /mnt/lustre_d247d.sanity
[11449.661516] Lustre: Unmounted lustre-client
[11449.662383] Lustre: Skipped 2 previous similar messages
[11449.668687] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre/d247d.sanity /mnt/lustre_d247d.sanity
[11449.769317] Lustre: DEBUG MARKER: grep -c /mnt/lustre_d247d.sanity' ' /proc/mounts
[11449.782121] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_d247d.sanity
[11449.954979] Lustre: DEBUG MARKER: umount  /mnt/lustre_d247d.sanity 2>&1
[11451.883912] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247e: mount .. as fileset ================= 02:41:55 \(1734403315\)
[11452.247864] Lustre: DEBUG MARKER: == sanity test 247e: mount .. as fileset ================= 02:41:55 (1734403315)
[11452.273533] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247e.sanity
[11452.285384] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre_d247e.sanity
[11452.341851] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[11452.355970] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_d247e.sanity//..
[11452.368946] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_d247e.sanity' ' /proc/mounts ||
[11452.368946] 			umount /mnt/lustre_d247e.sanity
[11452.534779] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre/.. /mnt/lustre_d247e.sanity
[11452.585544] LustreError: lustre-MDT0000-mdc-ffff988c74e1d800: operation mds_get_root to node 10.240.22.189@tcp failed: rc = -22
[11452.587755] LustreError: 660984:0:(llite_lib.c:710:client_common_fill_super()) cannot mds_connect: rc = -22
[11452.620657] LustreError: 660984:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -22
[11454.484665] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247f: mount striped or remote directory as fileset ========================================================== 02:41:57 \(1734403317\)
[11454.853483] Lustre: DEBUG MARKER: == sanity test 247f: mount striped or remote directory as fileset ========================================================== 02:41:57 (1734403317)
[11455.277818] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_247f needs \>= 2 MDTs
[11455.645426] Lustre: DEBUG MARKER: SKIP: sanity test_247f needs >= 2 MDTs
[11456.103465] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247g: striped directory submount revalidate ROOT from cache ========================================================== 02:41:59 \(1734403319\)
[11456.471911] Lustre: DEBUG MARKER: == sanity test 247g: striped directory submount revalidate ROOT from cache ========================================================== 02:41:59 (1734403319)
[11456.900621] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_247g needs \> 1 MDTs
[11457.268840] Lustre: DEBUG MARKER: SKIP: sanity test_247g needs > 1 MDTs
[11457.730172] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 247h: remote directory submount revalidate ROOT from cache ========================================================== 02:42:01 \(1734403321\)
[11458.101266] Lustre: DEBUG MARKER: == sanity test 247h: remote directory submount revalidate ROOT from cache ========================================================== 02:42:01 (1734403321)
[11458.524202] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_247h needs \> 1 MDTs
[11458.890187] Lustre: DEBUG MARKER: SKIP: sanity test_247h needs > 1 MDTs
[11459.342372] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 248a: fast read verification ============== 02:42:02 \(1734403322\)
[11459.711643] Lustre: DEBUG MARKER: == sanity test 248a: fast read verification ============== 02:42:02 (1734403322)
[11492.750616] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 248b: test short_io read and write for both small and large sizes ========================================================== 02:42:36 \(1734403356\)
[11493.126687] Lustre: DEBUG MARKER: == sanity test 248b: test short_io read and write for both small and large sizes ========================================================== 02:42:36 (1734403356)
[11527.965392] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 248c: verify whole file read behavior ===== 02:43:11 \(1734403391\)
[11528.336630] Lustre: DEBUG MARKER: == sanity test 248c: verify whole file read behavior ===== 02:43:11 (1734403391)
[11534.286035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 249: Write above 2T file size ============= 02:43:17 \(1734403397\)
[11534.656081] Lustre: DEBUG MARKER: == sanity test 249: Write above 2T file size ============= 02:43:17 (1734403397)
[11536.606150] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 250: Write above 16T limit ================ 02:43:20 \(1734403400\)
[11536.978642] Lustre: DEBUG MARKER: == sanity test 250: Write above 16T limit ================ 02:43:20 (1734403400)
[11538.888517] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 251a: Handling short read and write correctly ========================================================== 02:43:22 \(1734403402\)
[11539.253906] Lustre: DEBUG MARKER: == sanity test 251a: Handling short read and write correctly ========================================================== 02:43:22 (1734403402)
[11539.738351] Lustre: *** cfs_fail_loc=1407, val=0***
[11541.673470] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 251b: short read restore offset correctly ========================================================== 02:43:25 \(1734403405\)
[11542.041651] Lustre: DEBUG MARKER: == sanity test 251b: short read restore offset correctly ========================================================== 02:43:25 (1734403405)
[11542.072347] LustreError: 664945:0:(file.c:2305:do_file_read_iter()) cfs_fail_timeout id 1431 sleeping for 5000ms
[11547.168196] LustreError: 664945:0:(file.c:2305:do_file_read_iter()) cfs_fail_timeout id 1431 awake
[11549.035463] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 252: check lr_reader tool ================= 02:43:32 \(1734403412\)
[11549.405006] Lustre: DEBUG MARKER: == sanity test 252: check lr_reader tool ================= 02:43:32 (1734403412)
[11552.664091] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 253: Check object allocation limit ======== 02:43:36 \(1734403416\)
[11553.029375] Lustre: DEBUG MARKER: == sanity test 253: Check object allocation limit ======== 02:43:36 (1734403416)
[11560.658883] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_253 		2>/dev/null || echo foo
[11562.012159] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_253 |
[11562.012159] 			grep -e lustre-OST0000_UUID  | sort -u | tr '\n' ' ' 
[11563.032774] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_253 |
[11563.032774] 			grep -e lustre-OST0000_UUID  | sort -u | tr '\n' ' ' 
[11616.246914] Autotest: Test running for 190 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[11622.203037] Lustre: DEBUG MARKER: lctl get_param -n lov.lustre-*.pools.test_253 		2>/dev/null || echo foo
[11622.724150] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 254: Check changelog size ================= 02:44:46 \(1734403486\)
[11623.089355] Lustre: DEBUG MARKER: == sanity test 254: Check changelog size ================= 02:44:46 (1734403486)
[11627.955737] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 255a: check \'lfs ladvise -a willread\' ===== 02:44:51 \(1734403491\)
[11628.324334] Lustre: DEBUG MARKER: == sanity test 255a: check 'lfs ladvise -a willread' ===== 02:44:51 (1734403491)
[11917.024165] Autotest: Test running for 195 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[12217.898442] Autotest: Test running for 200 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[12518.808431] Autotest: Test running for 205 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[12524.445298] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 255b: check \'lfs ladvise -a dontneed\' ===== 02:59:47 \(1734404387\)
[12524.816091] Lustre: DEBUG MARKER: == sanity test 255b: check 'lfs ladvise -a dontneed' ===== 02:59:47 (1734404387)
[12539.989730] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 255c: suite of ladvise lockahead tests ==== 03:00:03 \(1734404403\)
[12540.355059] Lustre: DEBUG MARKER: == sanity test 255c: suite of ladvise lockahead tests ==== 03:00:03 (1734404403)
[12545.117140] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 256: Check llog delete for empty and not full state ========================================================== 03:00:08 \(1734404408\)
[12545.489712] Lustre: DEBUG MARKER: == sanity test 256: Check llog delete for empty and not full state ========================================================== 03:00:08 (1734404408)
[12553.312305] Lustre: lustre-MDT0000-mdc-ffff988c2f055800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12553.315722] Lustre: Skipped 2 previous similar messages
[12553.316918] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[12553.323092] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f173dbf6 to 0xdf05ec28f17e4399
[12553.326545] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[12553.328331] Lustre: Skipped 1 previous similar message
[12564.926236] Lustre: 432370:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734404412/real 1734404412]  req@ffff988d0782e700 x1818651235780864/t0(0) o400->lustre-MDT0000-mdc-ffff988c2f055800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734404428 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[12651.060433] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 257: xattr locks are not lost ============= 03:01:54 \(1734404514\)
[12651.428346] Lustre: DEBUG MARKER: == sanity test 257: xattr locks are not lost ============= 03:01:54 (1734404514)
[12651.864335] LustreError: lustre-MDT0000-mdc-ffff988c2f055800: operation ldlm_enqueue to node 10.240.22.189@tcp failed: rc = -14
[12655.710923] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[12655.714226] Lustre: lustre-MDT0000-mdc-ffff988c2f055800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12655.714779] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f17e4399 to 0xdf05ec28f17e4869
[12655.719947] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[12655.721732] Lustre: Skipped 1 previous similar message
[12658.549991] Lustre: DEBUG MARKER: lctl get_param -n at_max
[12674.584428] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 258a: verify i_mutex security behavior when suid attributes is set ========================================================== 03:02:18 \(1734404538\)
[12674.955396] Lustre: DEBUG MARKER: == sanity test 258a: verify i_mutex security behavior when suid attributes is set ========================================================== 03:02:18 (1734404538)
[12684.336748] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 258b: verify i_mutex security behavior ==== 03:02:27 \(1734404547\)
[12684.705260] Lustre: DEBUG MARKER: == sanity test 258b: verify i_mutex security behavior ==== 03:02:27 (1734404547)
[12687.877988] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 259: crash at delayed truncate ============ 03:02:31 \(1734404551\)
[12688.247874] Lustre: DEBUG MARKER: == sanity test 259: crash at delayed truncate ============ 03:02:31 (1734404551)
[12726.608196] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 260: Check mdc_close fail ================= 03:03:10 \(1734404590\)
[12726.971307] Lustre: DEBUG MARKER: == sanity test 260: Check mdc_close fail ================= 03:03:10 (1734404590)
[12727.101707] Lustre: *** cfs_fail_loc=806, val=0***
[12727.102689] Lustre: 675718:0:(mdc_request.c:922:mdc_close()) lustre-MDT0000-mdc-ffff988c2f055800: close of FID [0x2000061c3:0x142d:0x0] failed, file reference will be dropped when this client unmounts or is evicted
[12727.106045] LustreError: 675718:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff988c2f055800: inode [0x2000061c3:0x142d:0x0] mdc close failed: rc = -12
[12728.970067] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270a: DoM: basic functionality tests ====== 03:03:12 \(1734404592\)
[12729.338559] Lustre: DEBUG MARKER: == sanity test 270a: DoM: basic functionality tests ====== 03:03:12 (1734404592)
[12743.388186] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270b: DoM: maximum size overflow checks for DoM-only file ========================================================== 03:03:26 \(1734404606\)
[12743.755620] Lustre: DEBUG MARKER: == sanity test 270b: DoM: maximum size overflow checks for DoM-only file ========================================================== 03:03:26 (1734404606)
[12745.746445] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270c: DoM: DoM EA inheritance tests ======= 03:03:29 \(1734404609\)
[12746.114159] Lustre: DEBUG MARKER: == sanity test 270c: DoM: DoM EA inheritance tests ======= 03:03:29 (1734404609)
[12748.084100] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270d: DoM: change striping from DoM to RAID0 ========================================================== 03:03:31 \(1734404611\)
[12748.450578] Lustre: DEBUG MARKER: == sanity test 270d: DoM: change striping from DoM to RAID0 ========================================================== 03:03:31 (1734404611)
[12751.164262] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270e: DoM: lfs find with DoM files test === 03:03:34 \(1734404614\)
[12751.535977] Lustre: DEBUG MARKER: == sanity test 270e: DoM: lfs find with DoM files test === 03:03:34 (1734404614)
[12753.669718] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270f: DoM: maximum DoM stripe size checks ========================================================== 03:03:37 \(1734404617\)
[12754.037165] Lustre: DEBUG MARKER: == sanity test 270f: DoM: maximum DoM stripe size checks ========================================================== 03:03:37 (1734404617)
[12759.683502] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270g: DoM: default DoM stripe size depends on free space ========================================================== 03:03:43 \(1734404623\)
[12760.057739] Lustre: DEBUG MARKER: == sanity test 270g: DoM: default DoM stripe size depends on free space ========================================================== 03:03:43 (1734404623)
[12768.531035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270h: DoM: DoM stripe removal when disabled on server ========================================================== 03:03:52 \(1734404632\)
[12768.899513] Lustre: DEBUG MARKER: == sanity test 270h: DoM: DoM stripe removal when disabled on server ========================================================== 03:03:52 (1734404632)
[12773.736942] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270i: DoM: setting invalid DoM striping should fail ========================================================== 03:03:57 \(1734404637\)
[12774.105642] Lustre: DEBUG MARKER: == sanity test 270i: DoM: setting invalid DoM striping should fail ========================================================== 03:03:57 (1734404637)
[12776.017492] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 270j: DoM migration: DOM file to the OST-striped file \(plain\) ========================================================== 03:03:59 \(1734404639\)
[12776.386016] Lustre: DEBUG MARKER: == sanity test 270j: DoM migration: DOM file to the OST-striped file (plain) ========================================================== 03:03:59 (1734404639)
[12778.508977] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271a: DoM: data is cached for read after write ========================================================== 03:04:01 \(1734404641\)
[12778.881705] Lustre: DEBUG MARKER: == sanity test 271a: DoM: data is cached for read after write ========================================================== 03:04:01 (1734404641)
[12780.825581] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271b: DoM: no glimpse RPC for stat \(DoM only file\) ========================================================== 03:04:04 \(1734404644\)
[12781.196389] Lustre: DEBUG MARKER: == sanity test 271b: DoM: no glimpse RPC for stat (DoM only file) ========================================================== 03:04:04 (1734404644)
[12783.183945] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271ba: DoM: no glimpse RPC for stat \(combined file\) ========================================================== 03:04:06 \(1734404646\)
[12783.546241] Lustre: DEBUG MARKER: == sanity test 271ba: DoM: no glimpse RPC for stat (combined file) ========================================================== 03:04:06 (1734404646)
[12785.581936] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271c: DoM: IO lock at open saves enqueue RPCs ========================================================== 03:04:09 \(1734404649\)
[12785.945515] Lustre: DEBUG MARKER: == sanity test 271c: DoM: IO lock at open saves enqueue RPCs ========================================================== 03:04:09 (1734404649)
[12786.962716] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12788.122049] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12793.553168] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12794.708180] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12796.474766] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12797.644464] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12802.406007] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[12803.678251] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[12806.410824] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271d: DoM: read on open \(1K file in reply buffer\) ========================================================== 03:04:29 \(1734404669\)
[12806.777112] Lustre: DEBUG MARKER: == sanity test 271d: DoM: read on open (1K file in reply buffer) ========================================================== 03:04:29 (1734404669)
[12808.888507] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271f: DoM: read on open \(200K file and read tail\) ========================================================== 03:04:32 \(1734404672\)
[12809.258577] Lustre: DEBUG MARKER: == sanity test 271f: DoM: read on open (200K file and read tail) ========================================================== 03:04:32 (1734404672)
[12811.317337] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 271g: Discard DoM data vs client flush race ========================================================== 03:04:34 \(1734404674\)
[12811.688884] Lustre: DEBUG MARKER: == sanity test 271g: Discard DoM data vs client flush race ========================================================== 03:04:34 (1734404674)
[12812.729720] Lustre: *** cfs_fail_loc=314, val=0***
[12814.642652] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272a: DoM migration: new layout with the same DOM component ========================================================== 03:04:38 \(1734404678\)
[12815.009731] Lustre: DEBUG MARKER: == sanity test 272a: DoM migration: new layout with the same DOM component ========================================================== 03:04:38 (1734404678)
[12821.971708] Autotest: Test running for 210 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[12830.497412] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272b: DoM migration: DOM file to the OST-striped file \(plain\) ========================================================== 03:04:53 \(1734404693\)
[12830.865127] Lustre: DEBUG MARKER: == sanity test 272b: DoM migration: DOM file to the OST-striped file (plain) ========================================================== 03:04:53 (1734404693)
[12840.118775] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272c: DoM migration: DOM file to the OST-striped file \(composite\) ========================================================== 03:05:03 \(1734404703\)
[12840.487271] Lustre: DEBUG MARKER: == sanity test 272c: DoM migration: DOM file to the OST-striped file (composite) ========================================================== 03:05:03 (1734404703)
[12847.235835] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272d: DoM mirroring: OST-striped mirror to DOM file ========================================================== 03:05:10 \(1734404710\)
[12847.600637] Lustre: DEBUG MARKER: == sanity test 272d: DoM mirroring: OST-striped mirror to DOM file ========================================================== 03:05:10 (1734404710)
[12851.537052] LustreError: 686835:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff988c2f055800: inode [0x2000061c3:0x1c65:0x0] mdc close failed: rc = -22
[12853.784781] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272e: DoM mirroring: DOM mirror to the OST-striped file ========================================================== 03:05:17 \(1734404717\)
[12854.150344] Lustre: DEBUG MARKER: == sanity test 272e: DoM mirroring: DOM mirror to the OST-striped file ========================================================== 03:05:17 (1734404717)
[12862.861025] LustreError: 687311:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff988c2f055800: inode [0x2000061c3:0x1c69:0x0] mdc close failed: rc = -22
[12864.848216] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 272f: DoM migration: OST-striped file to DOM file ========================================================== 03:05:28 \(1734404728\)
[12865.213176] Lustre: DEBUG MARKER: == sanity test 272f: DoM migration: OST-striped file to DOM file ========================================================== 03:05:28 (1734404728)
[12871.358039] LustreError: 687776:0:(file.c:261:ll_close_inode_openhandle()) lustre-clilmv-ffff988c2f055800: inode [0x2000061c3:0x1c6e:0x0] mdc close failed: rc = -22
[12873.333650] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 273a: DoM: layout swapping should fail with DOM ========================================================== 03:05:36 \(1734404736\)
[12873.707220] Lustre: DEBUG MARKER: == sanity test 273a: DoM: layout swapping should fail with DOM ========================================================== 03:05:36 (1734404736)
[12875.887790] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 273b: DoM: race writeback and object destroy ========================================================== 03:05:39 \(1734404739\)
[12876.249734] Lustre: DEBUG MARKER: == sanity test 273b: DoM: race writeback and object destroy ========================================================== 03:05:39 (1734404739)
[12881.030621] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 273c: race writeback and object destroy === 03:05:44 \(1734404744\)
[12881.397969] Lustre: DEBUG MARKER: == sanity test 273c: race writeback and object destroy === 03:05:44 (1734404744)
[12884.115151] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 275: Read on a canceled duplicate lock ==== 03:05:47 \(1734404747\)
[12884.478515] Lustre: DEBUG MARKER: == sanity test 275: Read on a canceled duplicate lock ==== 03:05:47 (1734404747)
[12884.732384] LustreError: 668387:0:(ldlm_lockd.c:2927:ldlm_bl_thread_blwi()) cfs_fail_timeout id 31f sleeping for 4000ms
[12886.604365] LustreError: 668387:0:(ldlm_lockd.c:2927:ldlm_bl_thread_blwi()) cfs_fail_timeout interrupted
[12887.989617] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 276: Race between mount and obd_statfs ==== 03:05:51 \(1734404751\)
[12888.360061] Lustre: DEBUG MARKER: == sanity test 276: Race between mount and obd_statfs ==== 03:05:51 (1734404751)
[12891.230855] Lustre: lustre-OST0000-osc-ffff988c2f055800: Connection to lustre-OST0000 (at 10.240.22.182@tcp) was lost; in progress operations using this service will wait for recovery to complete
[12891.234084] Lustre: Skipped 1 previous similar message
[12951.113309] Lustre: lustre-OST0000-osc-ffff988c2f055800: Connection restored to 10.240.22.182@tcp (at 10.240.22.182@tcp)
[12951.115439] Lustre: Skipped 1 previous similar message
[13127.100503] Autotest: Test running for 215 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[13352.120813] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 277: Direct IO shall drop page cache ====== 03:13:35 \(1734405215\)
[13352.487259] Lustre: DEBUG MARKER: == sanity test 277: Direct IO shall drop page cache ====== 03:13:35 (1734405215)
[13356.482203] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 278: Race starting MDS between MDTs stop\/start ========================================================== 03:13:39 \(1734405219\)
[13356.850048] Lustre: DEBUG MARKER: == sanity test 278: Race starting MDS between MDTs stop/start ========================================================== 03:13:39 (1734405219)
[13357.276350] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_278 needs \>= 2 MDTs
[13357.640143] Lustre: DEBUG MARKER: SKIP: sanity test_278 needs >= 2 MDTs
[13358.093548] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 280: Race between MGS umount and client llog processing ========================================================== 03:13:41 \(1734405221\)
[13358.459283] Lustre: DEBUG MARKER: == sanity test 280: Race between MGS umount and client llog processing ========================================================== 03:13:41 (1734405221)
[13358.498544] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[13358.511169] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[13358.681991] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[13358.932960] Lustre: Unmounted lustre-client
[13358.933935] Lustre: Skipped 3 previous similar messages
[13359.249046] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[13359.261747] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[13369.434047] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[13369.436939] LustreError: MGC10.240.22.189@tcp: Confguration from log lustre-client failed from MGS -5. Communication error between node & MGS, a bad configuration, or other errors. See syslog for more info
[13369.440236] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f181b16a to 0xdf05ec28f181b2eb
[13369.449249] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[13369.450897] Lustre: Unmounted lustre-client
[13369.451031] Lustre: Skipped 2 previous similar messages
[13369.455301] LustreError: 696798:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -5
[13396.873894] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[13396.886226] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[13412.301971] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_connect to node 10.240.22.182@tcp failed: rc = -16
[13412.303290] Lustre: Mounted lustre-client
[13412.304994] Lustre: Skipped 4 previous similar messages
[13414.402327] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300a: basic striped dir sanity test ======= 03:14:37 \(1734405277\)
[13414.778474] Lustre: DEBUG MARKER: == sanity test 300a: basic striped dir sanity test ======= 03:14:37 (1734405277)
[13415.205359] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300a needs \>= 2 MDTs
[13415.574907] Lustre: DEBUG MARKER: SKIP: sanity test_300a needs >= 2 MDTs
[13416.036498] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300b: check ctime\/mtime for striped dir === 03:14:39 \(1734405279\)
[13416.408215] Lustre: DEBUG MARKER: == sanity test 300b: check ctime/mtime for striped dir === 03:14:39 (1734405279)
[13416.830272] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300b needs \>= 2 MDTs
[13417.194067] Lustre: DEBUG MARKER: SKIP: sanity test_300b needs >= 2 MDTs
[13417.311023] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_connect to node 10.240.22.182@tcp failed: rc = -16
[13417.650294] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300c: chown && check ls under striped directory ========================================================== 03:14:41 \(1734405281\)
[13418.018856] Lustre: DEBUG MARKER: == sanity test 300c: chown
[13418.446592] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300c needs \>= 2 MDTs
[13418.812271] Lustre: DEBUG MARKER: SKIP: sanity test_300c needs >= 2 MDTs
[13419.271544] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300d: check default stripe under striped directory ========================================================== 03:14:42 \(1734405282\)
[13419.636054] Lustre: DEBUG MARKER: == sanity test 300d: check default stripe under striped directory ========================================================== 03:14:42 (1734405282)
[13420.069998] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300d needs \>= 2 MDTs
[13420.437737] Lustre: DEBUG MARKER: SKIP: sanity test_300d needs >= 2 MDTs
[13420.897547] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300e: check rename under striped directory ========================================================== 03:14:44 \(1734405284\)
[13421.268892] Lustre: DEBUG MARKER: == sanity test 300e: check rename under striped directory ========================================================== 03:14:44 (1734405284)
[13421.715817] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300e needs \>= 2 MDTs
[13422.081877] Lustre: DEBUG MARKER: SKIP: sanity test_300e needs >= 2 MDTs
[13422.429577] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_connect to node 10.240.22.182@tcp failed: rc = -16
[13422.547118] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300f: check rename cross striped directory ========================================================== 03:14:46 \(1734405286\)
[13422.914692] Lustre: DEBUG MARKER: == sanity test 300f: check rename cross striped directory ========================================================== 03:14:46 (1734405286)
[13423.351517] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300f needs \>= 2 MDTs
[13423.713486] Lustre: DEBUG MARKER: SKIP: sanity test_300f needs >= 2 MDTs
[13424.168323] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300g: check default striped directory for normal directory ========================================================== 03:14:47 \(1734405287\)
[13424.535235] Lustre: DEBUG MARKER: == sanity test 300g: check default striped directory for normal directory ========================================================== 03:14:47 (1734405287)
[13424.966417] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300g needs \>= 2 MDTs
[13425.335014] Lustre: DEBUG MARKER: SKIP: sanity test_300g needs >= 2 MDTs
[13425.792036] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300h: check default striped directory for striped directory ========================================================== 03:14:49 \(1734405289\)
[13426.159884] Lustre: DEBUG MARKER: == sanity test 300h: check default striped directory for striped directory ========================================================== 03:14:49 (1734405289)
[13426.587430] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300h needs \>= 2 MDTs
[13426.960708] Lustre: DEBUG MARKER: SKIP: sanity test_300h needs >= 2 MDTs
[13427.420932] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300i: client handle unknown hash type striped directory ========================================================== 03:14:50 \(1734405290\)
[13427.555748] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_connect to node 10.240.22.182@tcp failed: rc = -16
[13427.791015] Lustre: DEBUG MARKER: == sanity test 300i: client handle unknown hash type striped directory ========================================================== 03:14:50 (1734405290)
[13428.229391] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300i needs \>= 2 MDTs
[13428.613422] Lustre: DEBUG MARKER: SKIP: sanity test_300i needs >= 2 MDTs
[13428.663992] Autotest: Test running for 220 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[13429.087134] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300j: test large update record ============ 03:14:52 \(1734405292\)
[13429.455124] Lustre: DEBUG MARKER: == sanity test 300j: test large update record ============ 03:14:52 (1734405292)
[13429.888439] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300j needs \>= 2 MDTs
[13430.257855] Lustre: DEBUG MARKER: SKIP: sanity test_300j needs >= 2 MDTs
[13430.718675] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300k: test large striped directory ======== 03:14:54 \(1734405294\)
[13431.088329] Lustre: DEBUG MARKER: == sanity test 300k: test large striped directory ======== 03:14:54 (1734405294)
[13431.514142] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300k needs \>= 2 MDTs
[13431.883590] Lustre: DEBUG MARKER: SKIP: sanity test_300k needs >= 2 MDTs
[13432.344332] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300l: non-root user to create dir under striped dir with stale layout ========================================================== 03:14:55 \(1734405295\)
[13432.670087] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_connect to node 10.240.22.182@tcp failed: rc = -16
[13432.717859] Lustre: DEBUG MARKER: == sanity test 300l: non-root user to create dir under striped dir with stale layout ========================================================== 03:14:55 (1734405295)
[13433.156798] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300l needs \>= 2 MDTs
[13433.521738] Lustre: DEBUG MARKER: SKIP: sanity test_300l needs >= 2 MDTs
[13433.985950] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300m: setstriped directory on single MDT FS ========================================================== 03:14:57 \(1734405297\)
[13434.351040] Lustre: DEBUG MARKER: == sanity test 300m: setstriped directory on single MDT FS ========================================================== 03:14:57 (1734405297)
[13436.329083] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300n: non-root user to create dir under striped dir with default EA ========================================================== 03:14:59 \(1734405299\)
[13436.697704] Lustre: DEBUG MARKER: == sanity test 300n: non-root user to create dir under striped dir with default EA ========================================================== 03:14:59 (1734405299)
[13437.124456] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300n needs \>= 2 MDTs
[13437.505127] Lustre: DEBUG MARKER: SKIP: sanity test_300n needs >= 2 MDTs
[13437.960263] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300o skipping SLOW test 300o
[13438.327782] Lustre: DEBUG MARKER: SKIP: sanity test_300o skipping SLOW test 300o
[13438.773912] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300p: create striped directory without space ========================================================== 03:15:02 \(1734405302\)
[13439.148282] Lustre: DEBUG MARKER: == sanity test 300p: create striped directory without space ========================================================== 03:15:02 (1734405302)
[13439.572138] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300p needs \>= 2 MDTs
[13439.938173] Lustre: DEBUG MARKER: SKIP: sanity test_300p needs >= 2 MDTs
[13440.397934] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300q: create remote directory under orphan directory ========================================================== 03:15:03 \(1734405303\)
[13440.766884] Lustre: DEBUG MARKER: == sanity test 300q: create remote directory under orphan directory ========================================================== 03:15:03 (1734405303)
[13441.194164] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300q needs \>= 2 MDTs
[13441.570565] Lustre: DEBUG MARKER: SKIP: sanity test_300q needs >= 2 MDTs
[13442.026373] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300r: test -1 striped directory =========== 03:15:05 \(1734405305\)
[13442.390163] Lustre: DEBUG MARKER: == sanity test 300r: test -1 striped directory =========== 03:15:05 (1734405305)
[13442.811500] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300r needs \>= 2 MDTs
[13443.176809] Lustre: DEBUG MARKER: SKIP: sanity test_300r needs >= 2 MDTs
[13443.633109] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300s: test lfs mkdir -c without -i ======== 03:15:07 \(1734405307\)
[13444.001461] Lustre: DEBUG MARKER: == sanity test 300s: test lfs mkdir -c without -i ======== 03:15:07 (1734405307)
[13444.424286] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300s needs \>= 2 MDTs
[13444.788576] Lustre: DEBUG MARKER: SKIP: sanity test_300s needs >= 2 MDTs
[13445.246927] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300t: test max_mdt_stripecount ============ 03:15:08 \(1734405308\)
[13445.617260] Lustre: DEBUG MARKER: == sanity test 300t: test max_mdt_stripecount ============ 03:15:08 (1734405308)
[13446.048169] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300t needs at least 2 MDTs
[13446.412960] Lustre: DEBUG MARKER: SKIP: sanity test_300t needs at least 2 MDTs
[13447.557236] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ua: basic overstriped dir sanity test == 03:15:11 \(1734405311\)
[13447.923927] Lustre: DEBUG MARKER: == sanity test 300ua: basic overstriped dir sanity test == 03:15:11 (1734405311)
[13448.348354] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300ua needs \>= 2 MDTs
[13448.714335] Lustre: DEBUG MARKER: SKIP: sanity test_300ua needs >= 2 MDTs
[13449.176545] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ub: test MDT overstriping interface & limits ========================================================== 03:15:12 \(1734405312\)
[13449.545020] Lustre: DEBUG MARKER: == sanity test 300ub: test MDT overstriping interface
[13449.972307] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300ub needs \>= 2 MDTs
[13450.346856] Lustre: DEBUG MARKER: SKIP: sanity test_300ub needs >= 2 MDTs
[13450.814550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300uc: test MDT overstriping as default & inheritance ========================================================== 03:15:14 \(1734405314\)
[13451.183551] Lustre: DEBUG MARKER: == sanity test 300uc: test MDT overstriping as default
[13451.605497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300uc needs \>= 2 MDTs
[13451.966685] Lustre: DEBUG MARKER: SKIP: sanity test_300uc needs >= 2 MDTs
[13452.426290] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ud: dir split ========================== 03:15:15 \(1734405315\)
[13452.805978] Lustre: DEBUG MARKER: == sanity test 300ud: dir split ========================== 03:15:15 (1734405315)
[13453.229807] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300ud needs \>= 2 MDTs
[13453.599265] Lustre: DEBUG MARKER: SKIP: sanity test_300ud needs >= 2 MDTs
[13454.058645] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ue: dir merge ========================== 03:15:17 \(1734405317\)
[13454.426966] Lustre: DEBUG MARKER: == sanity test 300ue: dir merge ========================== 03:15:17 (1734405317)
[13454.850017] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300ue needs \>= 2 MDTs
[13455.216823] Lustre: DEBUG MARKER: SKIP: sanity test_300ue needs >= 2 MDTs
[13455.673405] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300uf: migrate with too many local locks == 03:15:19 \(1734405319\)
[13456.041532] Lustre: DEBUG MARKER: == sanity test 300uf: migrate with too many local locks == 03:15:19 (1734405319)
[13456.464885] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300uf needs \>= 2 MDTs
[13456.831808] Lustre: DEBUG MARKER: SKIP: sanity test_300uf needs >= 2 MDTs
[13457.299254] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ug: migrate overstriped dirs =========== 03:15:20 \(1734405320\)
[13457.678216] Lustre: DEBUG MARKER: == sanity test 300ug: migrate overstriped dirs =========== 03:15:20 (1734405320)
[13458.103753] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300ug needs \>= 2 MDTs
[13458.469663] Lustre: DEBUG MARKER: SKIP: sanity test_300ug needs >= 2 MDTs
[13458.945366] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300uh: overstripe tunable max_stripes_per_mdt ========================================================== 03:15:22 \(1734405322\)
[13459.311243] Lustre: DEBUG MARKER: == sanity test 300uh: overstripe tunable max_stripes_per_mdt ========================================================== 03:15:22 (1734405322)
[13459.760232] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_300uh needs \>= 2 MDTs
[13460.130284] Lustre: DEBUG MARKER: SKIP: sanity test_300uh needs >= 2 MDTs
[13460.587449] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 300ui: overstripe is not supported on one MDT system ========================================================== 03:15:24 \(1734405324\)
[13460.956779] Lustre: DEBUG MARKER: == sanity test 300ui: overstripe is not supported on one MDT system ========================================================== 03:15:24 (1734405324)
[13463.191429] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 310a: open unlink remote file ============= 03:15:26 \(1734405326\)
[13463.560254] Lustre: DEBUG MARKER: == sanity test 310a: open unlink remote file ============= 03:15:26 (1734405326)
[13463.982591] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_310a needs \>= 4 MDTs
[13464.347653] Lustre: DEBUG MARKER: SKIP: sanity test_310a needs >= 4 MDTs
[13464.818201] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 310b: unlink remote file with multiple links while open ========================================================== 03:15:28 \(1734405328\)
[13465.188469] Lustre: DEBUG MARKER: == sanity test 310b: unlink remote file with multiple links while open ========================================================== 03:15:28 (1734405328)
[13465.620650] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_310b needs \>= 4 MDTs
[13465.988859] Lustre: DEBUG MARKER: SKIP: sanity test_310b needs >= 4 MDTs
[13466.449872] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 310c: open-unlink remote file with multiple links ========================================================== 03:15:29 \(1734405329\)
[13466.816713] Lustre: DEBUG MARKER: == sanity test 310c: open-unlink remote file with multiple links ========================================================== 03:15:29 (1734405329)
[13467.243323] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_310c needs \>= 4 MDTs
[13467.614356] Lustre: DEBUG MARKER: SKIP: sanity test_310c needs >= 4 MDTs
[13468.075142] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 311: disable OSP precreate, and unlink should destroy objs ========================================================== 03:15:31 \(1734405331\)
[13468.443452] Lustre: DEBUG MARKER: == sanity test 311: disable OSP precreate, and unlink should destroy objs ========================================================== 03:15:31 (1734405331)
[13469.228176] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13470.804291] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13473.235773] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[13474.180677] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[13488.717145] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 312: make sure ZFS adjusts its block size by write pattern ========================================================== 03:15:52 \(1734405352\)
[13489.088268] Lustre: DEBUG MARKER: == sanity test 312: make sure ZFS adjusts its block size by write pattern ========================================================== 03:15:52 (1734405352)
[13489.579047] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_312 the test only applies to zfs
[13489.948124] Lustre: DEBUG MARKER: SKIP: sanity test_312 the test only applies to zfs
[13490.418203] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 313: io should fail after last_rcvd update fail ========================================================== 03:15:53 \(1734405353\)
[13490.786022] Lustre: DEBUG MARKER: == sanity test 313: io should fail after last_rcvd update fail ========================================================== 03:15:53 (1734405353)
[13493.454827] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 314: OSP shouldn\'t fail after last_rcvd update failure ========================================================== 03:15:56 \(1734405356\)
[13493.822080] Lustre: DEBUG MARKER: == sanity test 314: OSP shouldn't fail after last_rcvd update failure ========================================================== 03:15:56 (1734405356)
[13508.668577] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 315: read should be accounted ============= 03:16:12 \(1734405372\)
[13509.037676] Lustre: DEBUG MARKER: == sanity test 315: read should be accounted ============= 03:16:12 (1734405372)
[13515.350913] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 316: lfs migrate of file with large_xattr enabled ========================================================== 03:16:18 \(1734405378\)
[13515.720383] Lustre: DEBUG MARKER: == sanity test 316: lfs migrate of file with large_xattr enabled ========================================================== 03:16:18 (1734405378)
[13516.142711] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_316 needs \>= 2 MDTs
[13516.508203] Lustre: DEBUG MARKER: SKIP: sanity test_316 needs >= 2 MDTs
[13516.964972] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 317: Verify blocks get correctly update after truncate ========================================================== 03:16:20 \(1734405380\)
[13517.331886] Lustre: DEBUG MARKER: == sanity test 317: Verify blocks get correctly update after truncate ========================================================== 03:16:20 (1734405380)
[13522.108980] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 318: Verify async readahead tunables ====== 03:16:25 \(1734405385\)
[13522.479664] Lustre: DEBUG MARKER: == sanity test 318: Verify async readahead tunables ====== 03:16:25 (1734405385)
[13522.527516] LustreError: 710397:0:(lproc_llite.c:1760:read_ahead_async_file_threshold_mb_store()) lustre: can't set read_ahead_async_file_threshold_mb=65 > max_read_readahead_per_file_mb=64
[13524.410744] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 319: lost lease lock on migrate error ===== 03:16:27 \(1734405387\)
[13524.779694] Lustre: DEBUG MARKER: == sanity test 319: lost lease lock on migrate error ===== 03:16:27 (1734405387)
[13530.213687] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_319 needs \>= 2 MDTs
[13530.581259] Lustre: DEBUG MARKER: SKIP: sanity test_319 needs >= 2 MDTs
[13531.047916] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 350: force NID mismatch path to be exercised ========================================================== 03:16:34 \(1734405394\)
[13536.430951] Lustre: DEBUG MARKER: == sanity test 350: force NID mismatch path to be exercised ========================================================== 03:16:34 (1734405394)
[13575.089987] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 360: ldiskfs unlink in a separate thread == 03:17:18 \(1734405438\)
[13575.461515] Lustre: DEBUG MARKER: == sanity test 360: ldiskfs unlink in a separate thread == 03:17:18 (1734405438)
[13637.080534] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398a: direct IO should cancel lock otherwise lockless ========================================================== 03:18:20 \(1734405500\)
[13637.449136] Lustre: DEBUG MARKER: == sanity test 398a: direct IO should cancel lock otherwise lockless ========================================================== 03:18:20 (1734405500)
[13643.235960] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398b: DIO and buffer IO race ============== 03:18:26 \(1734405506\)
[13643.613884] Lustre: DEBUG MARKER: == sanity test 398b: DIO and buffer IO race ============== 03:18:26 (1734405506)
[13729.403744] Autotest: Test running for 225 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[14030.211405] Autotest: Test running for 230 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[14331.020425] Autotest: Test running for 235 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[14631.990381] Autotest: Test running for 240 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[14932.805106] Autotest: Test running for 245 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15233.829334] Autotest: Test running for 250 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15400.158304] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param osc.lustre-OST*-osc-*/state
[15402.299201] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398c: run fio to test AIO ================= 03:47:45 \(1734407265\)
[15402.672646] Lustre: DEBUG MARKER: == sanity test 398c: run fio to test AIO ================= 03:47:45 (1734407265)
[15534.635383] Autotest: Test running for 255 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15619.053951] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398d: run aiocp to verify block size \> stripe size ========================================================== 03:51:22 \(1734407482\)
[15619.423952] Lustre: DEBUG MARKER: == sanity test 398d: run aiocp to verify block size > stripe size ========================================================== 03:51:22 (1734407482)
[15629.675094] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398e: O_Direct open cleared by fcntl doesn\'t cause hang ========================================================== 03:51:33 \(1734407493\)
[15630.043422] Lustre: DEBUG MARKER: == sanity test 398e: O_Direct open cleared by fcntl doesn't cause hang ========================================================== 03:51:33 (1734407493)
[15632.038021] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398f: verify aio handles ll_direct_rw_pages errors correctly ========================================================== 03:51:35 \(1734407495\)
[15632.405497] Lustre: DEBUG MARKER: == sanity test 398f: verify aio handles ll_direct_rw_pages errors correctly ========================================================== 03:51:35 (1734407495)
[15636.221516] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398g: verify parallel dio async RPC submission ========================================================== 03:51:39 \(1734407499\)
[15636.591966] Lustre: DEBUG MARKER: == sanity test 398g: verify parallel dio async RPC submission ========================================================== 03:51:39 (1734407499)
[15660.565393] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398h: verify correctness of read & write with i\/o size \>\> stripe size ========================================================== 03:52:04 \(1734407524\)
[15660.931048] Lustre: DEBUG MARKER: == sanity test 398h: verify correctness of read
[15672.461551] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398i: verify parallel dio handles ll_direct_rw_pages errors correctly ========================================================== 03:52:15 \(1734407535\)
[15672.833332] Lustre: DEBUG MARKER: == sanity test 398i: verify parallel dio handles ll_direct_rw_pages errors correctly ========================================================== 03:52:15 (1734407535)
[15676.305164] Lustre: *** cfs_fail_loc=1418, val=0***
[15678.218385] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398j: test parallel dio where stripe size \> rpc_size ========================================================== 03:52:21 \(1734407541\)
[15678.586886] Lustre: DEBUG MARKER: == sanity test 398j: test parallel dio where stripe size > rpc_size ========================================================== 03:52:21 (1734407541)
[15688.605846] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398k: test enospc on first stripe ========= 03:52:32 \(1734407552\)
[15688.978439] Lustre: DEBUG MARKER: == sanity test 398k: test enospc on first stripe ========= 03:52:32 (1734407552)
[15701.587778] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_398k 49715204 \> 2100000 skipping out-of-space test on OST0
[15702.062945] Lustre: DEBUG MARKER: SKIP: sanity test_398k 49715204 > 2100000 skipping out-of-space test on OST0
[15702.526024] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398l: test enospc on intermediate stripe\/RPC ========================================================== 03:52:46 \(1734407566\)
[15702.895347] Lustre: DEBUG MARKER: == sanity test 398l: test enospc on intermediate stripe/RPC ========================================================== 03:52:46 (1734407566)
[15709.029256] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_398l 49692148 \> 2100000 skipping out-of-space test on OST0
[15709.398420] Lustre: DEBUG MARKER: SKIP: sanity test_398l 49692148 > 2100000 skipping out-of-space test on OST0
[15723.482790] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398m: test RPC failures with parallel dio ========================================================== 03:53:06 \(1734407586\)
[15723.947615] Lustre: DEBUG MARKER: == sanity test 398m: test RPC failures with parallel dio ========================================================== 03:53:06 (1734407586)
[15724.370919] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[15724.373108] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d098a3740 x1818651256134144/t0(0) o4->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407605 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'dd.0' uid:0 gid:0
[15725.388487] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[15725.390608] LustreError: Skipped 3 previous similar messages
[15725.391775] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d098a3400 x1818651256134528/t0(0) o4->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407606 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_00.0' uid:0 gid:0
[15725.396808] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 3 previous similar messages
[15727.404481] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[15727.406598] LustreError: Skipped 3 previous similar messages
[15727.407742] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d098a3740 x1818651256136832/t0(0) o4->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407608 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_00.0' uid:0 gid:0
[15727.412755] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 3 previous similar messages
[15730.511725] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d098a3400 x1818651256138496/t0(0) o4->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407611 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_01_00.0' uid:0 gid:0
[15730.516916] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 5 previous similar messages
[15734.604753] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[15734.604844] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d05dad380 x1818651256139264/t0(0) o4->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407615 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_01.0' uid:0 gid:0
[15734.606918] LustreError: Skipped 8 previous similar messages
[15745.868381] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[15745.868381] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[15745.868385] LustreError: Skipped 4 previous similar messages
[15745.870537] LustreError: Skipped 4 previous similar messages
[15745.870603] LustreError: 432369:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d05dae080 x1818651256142080/t0(0) o4->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407626 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_00.0' uid:0 gid:0
[15745.879826] LustreError: 432369:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 8 previous similar messages
[15769.420486] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d05daed80 x1818651256146304/t0(0) o4->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407650 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_01.0' uid:0 gid:0
[15769.425607] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 10 previous similar messages
[15779.436100] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_write to node 10.240.22.182@tcp failed: rc = -5
[15779.438303] LustreError: Skipped 14 previous similar messages
[15779.439470] LustreError: 432370:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff988d04991000: too many resent retries for object: 9663685440:2843: rc = -5
[15780.427964] LustreError: 432371:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff988d04991000: too many resent retries for object: 9663685440:2843: rc = -5
[15780.430793] LustreError: 432371:0:(osc_request.c:2602:brw_interpret()) Skipped 2 previous similar messages
[15801.932469] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d05dac000 x1818651256175744/t0(0) o3->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/440 e 0 to 0 dl 1734407681 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_01.0' uid:0 gid:0
[15801.937619] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 19 previous similar messages
[15835.444377] Autotest: Test running for 260 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[15848.012184] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_read to node 10.240.22.182@tcp failed: rc = -5
[15848.014271] LustreError: Skipped 43 previous similar messages
[15848.015445] LustreError: 432370:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff988d04991000: too many resent retries for object: 9663685440:2844: rc = -5
[15869.771923] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d05dae700 x1818651256198784/t0(0) o4->lustre-OST0001-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/224 e 0 to 0 dl 1734407749 ref 2 fl Interpret:ReMQU/200/0 rc -5/-5 job:'ptlrpcd_00_01.0' uid:0 gid:0
[15869.777057] LustreError: 432370:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 47 previous similar messages
[15903.307368] LustreError: 432370:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff988d04991000: too many resent retries for object: 10737423264:1960: rc = -5
[15903.310188] LustreError: 432370:0:(osc_request.c:2602:brw_interpret()) Skipped 3 previous similar messages
[15961.931203] LustreError: 432369:0:(osc_request.c:2602:brw_interpret()) lustre-OST0001-osc-ffff988d04991000: too many resent retries for object: 10737423264:1961: rc = -5
[15961.933980] LustreError: 432369:0:(osc_request.c:2602:brw_interpret()) Skipped 3 previous similar messages
[15965.974991] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398n: test append with parallel DIO ======= 03:57:09 \(1734407829\)
[15966.340917] Lustre: DEBUG MARKER: == sanity test 398n: test append with parallel DIO ======= 03:57:09 (1734407829)
[15975.701279] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398o: right kms with DIO ================== 03:57:19 \(1734407839\)
[15976.076016] Lustre: DEBUG MARKER: == sanity test 398o: right kms with DIO ================== 03:57:19 (1734407839)
[15978.345353] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398p: race aio with buffered i\/o ========== 03:57:21 \(1734407841\)
[15978.712352] Lustre: DEBUG MARKER: == sanity test 398p: race aio with buffered i/o ========== 03:57:21 (1734407841)
[16024.569831] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398q: race dio with buffered i\/o ========== 03:58:08 \(1734407888\)
[16024.940253] Lustre: DEBUG MARKER: == sanity test 398q: race dio with buffered i/o ========== 03:58:08 (1734407888)
[16076.183507] Autotest: Test running for 265 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[16207.101114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398r: i\/o error on file read ============== 04:01:10 \(1734408070\)
[16207.468415] Lustre: DEBUG MARKER: == sanity test 398r: i/o error on file read ============== 04:01:10 (1734408070)
[16207.861669] LustreError: lustre-OST0000-osc-ffff988d04991000: operation ost_read to node 10.240.22.182@tcp failed: rc = -5
[16207.863786] LustreError: Skipped 91 previous similar messages
[16207.864923] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) @@@ redo for recoverable error -5  req@ffff988d098a1d40 x1818651262131968/t0(0) o3->lustre-OST0000-osc-ffff988d04991000@10.240.22.182@tcp:6/4 lens 488/4536 e 0 to 0 dl 1734408087 ref 2 fl Interpret:RMQU/200/0 rc -5/-5 job:'cat.0' uid:0 gid:0
[16207.869777] LustreError: 432371:0:(osc_request.c:2448:osc_brw_redo_request()) Skipped 55 previous similar messages
[16264.009617] LustreError: 432371:0:(osc_request.c:2602:brw_interpret()) lustre-OST0000-osc-ffff988d04991000: too many resent retries for object: 9663685440:2847: rc = -5
[16264.012431] LustreError: 432371:0:(osc_request.c:2602:brw_interpret()) Skipped 3 previous similar messages
[16265.995761] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 398s: i\/o error on mirror file read ======= 04:02:09 \(1734408129\)
[16266.363582] Lustre: DEBUG MARKER: == sanity test 398s: i/o error on mirror file read ======= 04:02:09 (1734408129)
[16268.825896] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 399a: fake write should not be slower than normal write ========================================================== 04:02:12 \(1734408132\)
[16269.199215] Lustre: DEBUG MARKER: == sanity test 399a: fake write should not be slower than normal write ========================================================== 04:02:12 (1734408132)
[16377.004800] Autotest: Test running for 270 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[16385.230129] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 399b: fake read should not be slower than normal read ========================================================== 04:04:08 \(1734408248\)
[16385.609014] Lustre: DEBUG MARKER: == sanity test 399b: fake read should not be slower than normal read ========================================================== 04:04:08 (1734408248)
[16393.687417] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 400a: Lustre client api program can compile and link ========================================================== 04:04:17 \(1734408257\)
[16394.063350] Lustre: DEBUG MARKER: == sanity test 400a: Lustre client api program can compile and link ========================================================== 04:04:17 (1734408257)
[16396.331386] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 400b: packaged headers can be compiled ==== 04:04:19 \(1734408259\)
[16396.700913] Lustre: DEBUG MARKER: == sanity test 400b: packaged headers can be compiled ==== 04:04:19 (1734408259)
[16398.915359] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401a: Verify if \'lctl list_param -R\' can list parameters recursively ========================================================== 04:04:22 \(1734408262\)
[16399.289596] Lustre: DEBUG MARKER: == sanity test 401a: Verify if 'lctl list_param -R' can list parameters recursively ========================================================== 04:04:22 (1734408262)
[16401.252468] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401aa: Verify that \'lctl list_param -p\' lists the correct path names ========================================================== 04:04:24 \(1734408264\)
[16401.625441] Lustre: DEBUG MARKER: == sanity test 401aa: Verify that 'lctl list_param -p' lists the correct path names ========================================================== 04:04:24 (1734408264)
[16403.535298] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401b: Verify \'lctl {get,set}_param\' continue after error ========================================================== 04:04:27 \(1734408267\)
[16403.925657] Lustre: DEBUG MARKER: == sanity test 401b: Verify 'lctl get_param' set_param' continue after error ========================================================== 04:04:27 (1734408267)
[16405.878110] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401c: Verify \'lctl set_param\' without value fails in either format. ========================================================== 04:04:29 \(1734408269\)
[16406.255533] Lustre: DEBUG MARKER: == sanity test 401c: Verify 'lctl set_param' without value fails in either format. ========================================================== 04:04:29 (1734408269)
[16408.202079] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401d: Verify \'lctl set_param\' accepts values containing \'=\' ========================================================== 04:04:31 \(1734408271\)
[16408.571244] Lustre: DEBUG MARKER: == sanity test 401d: Verify 'lctl set_param' accepts values containing '=' ========================================================== 04:04:31 (1734408271)
[16410.534200] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401db: Verify \'lctl set_param\' does not add trailing \'=\' ========================================================== 04:04:34 \(1734408274\)
[16410.904979] Lustre: DEBUG MARKER: == sanity test 401db: Verify 'lctl set_param' does not add trailing '=' ========================================================== 04:04:34 (1734408274)
[16411.238688] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16412.256468] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16413.274086] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16413.602265] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16414.620517] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16415.638275] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16416.656428] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16417.674142] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16418.692134] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16419.709893] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16420.727467] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16421.745568] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16422.763672] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16423.781602] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16424.799189] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16425.816753] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16426.834284] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16427.852218] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16428.869815] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16429.888036] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16430.905734] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16431.923262] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16432.940928] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16433.958469] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16434.976166] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16435.993858] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16437.011422] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16438.028922] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16439.046344] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16440.063656] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16441.080274] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16442.097174] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16443.114335] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16444.131773] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16445.149304] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16446.167990] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16447.186407] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16448.204069] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16449.221599] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16450.239145] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16451.256846] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16452.276308] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16453.293855] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16454.311649] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16455.329560] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16456.348622] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16457.366984] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16458.384836] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16459.402664] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16460.420184] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16461.437786] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16462.455707] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16463.473287] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16464.491172] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16465.509048] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16466.526718] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16467.544551] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16468.562704] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16469.580799] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16470.598438] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16471.616370] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16472.634208] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16473.652882] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16474.672050] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16475.691069] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16476.710105] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16477.731965] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16478.754967] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16479.773276] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16480.792364] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16481.811620] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16482.829473] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16483.847306] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16484.865186] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16485.882922] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16486.900301] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16487.917706] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16488.935174] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16489.952721] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16490.969460] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16491.985517] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16493.001905] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16494.018257] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16495.034483] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16496.051340] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16497.068786] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16498.085550] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16499.102428] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16500.120295] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16501.138188] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16502.156022] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16503.173835] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16506.106794] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16507.127680] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16508.145711] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n at_min
[16508.634793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401e: verify \'lctl get_param\' works with NID in parameter ========================================================== 04:06:12 \(1734408372\)
[16509.023395] Lustre: DEBUG MARKER: == sanity test 401e: verify 'lctl get_param' works with NID in parameter ========================================================== 04:06:12 (1734408372)
[16510.958822] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 401f: check \'lctl list_param\' doesn\'t follow symlinks with --no-links ========================================================== 04:06:14 \(1734408374\)
[16511.329860] Lustre: DEBUG MARKER: == sanity test 401f: check 'lctl list_param' doesn't follow symlinks with --no-links ========================================================== 04:06:14 (1734408374)
[16513.321692] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 402: Return ENOENT to lod_generate_and_set_lovea ========================================================== 04:06:16 \(1734408376\)
[16513.695758] Lustre: DEBUG MARKER: == sanity test 402: Return ENOENT to lod_generate_and_set_lovea ========================================================== 04:06:16 (1734408376)
[16516.020410] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 403: i_nlink should not drop to zero due to aliasing ========================================================== 04:06:19 \(1734408379\)
[16516.395766] Lustre: DEBUG MARKER: == sanity test 403: i_nlink should not drop to zero due to aliasing ========================================================== 04:06:19 (1734408379)
[16516.563566] sysctl (732140): drop_caches: 2
[16518.480519] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 404: validate manual {de}activated works properly for OSPs ========================================================== 04:06:21 \(1734408381\)
[16518.849806] Lustre: DEBUG MARKER: == sanity test 404: validate manual {de}activated works properly for OSPs ========================================================== 04:06:21 (1734408381)
[16529.731553] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 405: Various layout swap lock tests ======= 04:06:33 \(1734408393\)
[16530.101351] Lustre: DEBUG MARKER: == sanity test 405: Various layout swap lock tests ======= 04:06:33 (1734408393)
[16540.201141] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_405 layout swap does not support DOM files so far
[16540.569453] Lustre: DEBUG MARKER: SKIP: sanity test_405 layout swap does not support DOM files so far
[16541.033212] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 406: DNE support fs default striping ====== 04:06:44 \(1734408404\)
[16541.404831] Lustre: DEBUG MARKER: == sanity test 406: DNE support fs default striping ====== 04:06:44 (1734408404)
[16541.832784] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_406 needs \>= 2 MDTs
[16542.202765] Lustre: DEBUG MARKER: SKIP: sanity test_406 needs >= 2 MDTs
[16542.648178] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_407 skipping ALWAYS excluded test 407
[16543.018267] Lustre: DEBUG MARKER: SKIP: sanity test_407 skipping ALWAYS excluded test 407
[16543.469524] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 408: drop_caches should not hang due to page leaks ========================================================== 04:06:46 \(1734408406\)
[16543.840917] Lustre: DEBUG MARKER: == sanity test 408: drop_caches should not hang due to page leaks ========================================================== 04:06:46 (1734408406)
[16543.948067] Lustre: *** cfs_fail_loc=40a, val=0***
[16543.949093] LustreError: 432371:0:(osc_request.c:2870:osc_build_rpc()) lustre-OST0003-osc-ffff988d04991000: prep_req failed: rc = -22
[16543.951325] LustreError: 432371:0:(osc_cache.c:2153:osc_check_rpcs()) Read request failed with -22
[16545.859856] bash (733903): drop_caches: 2
[16547.775072] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 409: Large amount of cross-MDTs hard links on the same file ========================================================== 04:06:51 \(1734408411\)
[16548.145334] Lustre: DEBUG MARKER: == sanity test 409: Large amount of cross-MDTs hard links on the same file ========================================================== 04:06:51 (1734408411)
[16548.570463] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_409 needs \>= 2 MDTs
[16548.938197] Lustre: DEBUG MARKER: SKIP: sanity test_409 needs >= 2 MDTs
[16549.403707] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 410: Test inode number returned from kernel thread ========================================================== 04:06:52 \(1734408412\)
[16549.770802] Lustre: DEBUG MARKER: == sanity test 410: Test inode number returned from kernel thread ========================================================== 04:06:52 (1734408412)
[16549.829495] lustre_kinode_3192: CONFIG_X86_X32 is set
[16549.832168] lustre_kinode_3192: inode is 144115708588985666
[16549.833550] lustre_kinode_3192: inode is 144115708588985666
[16549.834671] lustre_kinode_3192: inode numbers are identical: 144115708588985666
[16551.773250] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 411a: Slab allocation error with cgroup does not LBUG ========================================================== 04:06:55 \(1734408415\)
[16552.141057] Lustre: DEBUG MARKER: == sanity test 411a: Slab allocation error with cgroup does not LBUG ========================================================== 04:06:55 (1734408415)
[16552.566793] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_411a no setup for cgroup
[16552.933022] Lustre: DEBUG MARKER: SKIP: sanity test_411a no setup for cgroup
[16553.390233] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 411b: confirm Lustre can avoid OOM with reasonable cgroups limits ========================================================== 04:06:56 \(1734408416\)
[16553.755985] Lustre: DEBUG MARKER: == sanity test 411b: confirm Lustre can avoid OOM with reasonable cgroups limits ========================================================== 04:06:56 (1734408416)
[16554.182542] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_411b no setup for cgroup
[16554.549046] Lustre: DEBUG MARKER: SKIP: sanity test_411b no setup for cgroup
[16555.010442] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 412: mkdir on specific MDTs =============== 04:06:58 \(1734408418\)
[16555.377710] Lustre: DEBUG MARKER: == sanity test 412: mkdir on specific MDTs =============== 04:06:58 (1734408418)
[16555.805749] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_412 needs \>= 2 MDTs
[16556.177714] Lustre: DEBUG MARKER: SKIP: sanity test_412 needs >= 2 MDTs
[16556.637381] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413a: QoS mkdir with \'lfs mkdir -i -1\' ==== 04:07:00 \(1734408420\)
[16557.006146] Lustre: DEBUG MARKER: == sanity test 413a: QoS mkdir with 'lfs mkdir -i -1' ==== 04:07:00 (1734408420)
[16557.432213] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413a We need at least 2 MDTs for this test
[16557.802226] Lustre: DEBUG MARKER: SKIP: sanity test_413a We need at least 2 MDTs for this test
[16558.265160] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413b: QoS mkdir under dir whose default LMV starting MDT offset is -1 ========================================================== 04:07:01 \(1734408421\)
[16558.633619] Lustre: DEBUG MARKER: == sanity test 413b: QoS mkdir under dir whose default LMV starting MDT offset is -1 ========================================================== 04:07:01 (1734408421)
[16559.059219] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413b We need at least 2 MDTs for this test
[16559.424193] Lustre: DEBUG MARKER: SKIP: sanity test_413b We need at least 2 MDTs for this test
[16559.891102] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413c: mkdir with default LMV max inherit rr ========================================================== 04:07:03 \(1734408423\)
[16560.266097] Lustre: DEBUG MARKER: == sanity test 413c: mkdir with default LMV max inherit rr ========================================================== 04:07:03 (1734408423)
[16560.690427] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413c We need at least 2 MDTs for this test
[16561.057746] Lustre: DEBUG MARKER: SKIP: sanity test_413c We need at least 2 MDTs for this test
[16561.521313] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413d: inherit ROOT default LMV ============ 04:07:05 \(1734408425\)
[16561.890471] Lustre: DEBUG MARKER: == sanity test 413d: inherit ROOT default LMV ============ 04:07:05 (1734408425)
[16562.317820] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413d We need at least 2 MDTs for this test
[16562.688773] Lustre: DEBUG MARKER: SKIP: sanity test_413d We need at least 2 MDTs for this test
[16563.152738] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413e: check default max-inherit value ===== 04:07:06 \(1734408426\)
[16563.520821] Lustre: DEBUG MARKER: == sanity test 413e: check default max-inherit value ===== 04:07:06 (1734408426)
[16563.951114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413e We need at least 2 MDTs for this test
[16564.319660] Lustre: DEBUG MARKER: SKIP: sanity test_413e We need at least 2 MDTs for this test
[16564.780494] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413f: lfs getdirstripe -D list ROOT default LMV if it\'s not set on dir ========================================================== 04:07:08 \(1734408428\)
[16565.147177] Lustre: DEBUG MARKER: == sanity test 413f: lfs getdirstripe -D list ROOT default LMV if it's not set on dir ========================================================== 04:07:08 (1734408428)
[16565.571322] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413f We need at least 2 MDTs for this test
[16565.935018] Lustre: DEBUG MARKER: SKIP: sanity test_413f We need at least 2 MDTs for this test
[16566.397665] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413g: enforce ROOT default LMV on subdir mount ========================================================== 04:07:09 \(1734408429\)
[16566.769043] Lustre: DEBUG MARKER: == sanity test 413g: enforce ROOT default LMV on subdir mount ========================================================== 04:07:09 (1734408429)
[16567.199661] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413g We need at least 2 MDTs for this test
[16567.568586] Lustre: DEBUG MARKER: SKIP: sanity test_413g We need at least 2 MDTs for this test
[16568.029413] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413h: don\'t stick to parent for round-robin dirs ========================================================== 04:07:11 \(1734408431\)
[16568.398900] Lustre: DEBUG MARKER: == sanity test 413h: don't stick to parent for round-robin dirs ========================================================== 04:07:11 (1734408431)
[16568.829383] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413h We need at least 2 MDTs for this test
[16569.197738] Lustre: DEBUG MARKER: SKIP: sanity test_413h We need at least 2 MDTs for this test
[16569.664683] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413i: check default layout inheritance ==== 04:07:13 \(1734408433\)
[16570.035033] Lustre: DEBUG MARKER: == sanity test 413i: check default layout inheritance ==== 04:07:13 (1734408433)
[16570.461751] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413i needs \>= 2 MDTs
[16570.831203] Lustre: DEBUG MARKER: SKIP: sanity test_413i needs >= 2 MDTs
[16571.293785] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413j: set default LMV by setxattr ========= 04:07:14 \(1734408434\)
[16571.666669] Lustre: DEBUG MARKER: == sanity test 413j: set default LMV by setxattr ========= 04:07:14 (1734408434)
[16572.091318] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_413j needs \>= 2 MDTs
[16572.460784] Lustre: DEBUG MARKER: SKIP: sanity test_413j needs >= 2 MDTs
[16572.921906] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413k: QoS mkdir exclude prefixes ========== 04:07:16 \(1734408436\)
[16573.289919] Lustre: DEBUG MARKER: == sanity test 413k: QoS mkdir exclude prefixes ========== 04:07:16 (1734408436)
[16575.436713] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 413z: 413 test cleanup ==================== 04:07:18 \(1734408438\)
[16575.806377] Lustre: DEBUG MARKER: == sanity test 413z: 413 test cleanup ==================== 04:07:18 (1734408438)
[16577.742470] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 414: simulate ENOMEM in ptlrpc_register_bulk\(\) ========================================================== 04:07:21 \(1734408441\)
[16578.115182] Lustre: DEBUG MARKER: == sanity test 414: simulate ENOMEM in ptlrpc_register_bulk() ========================================================== 04:07:21 (1734408441)
[16578.212185] Lustre: *** cfs_fail_loc=521, val=0***
[16578.213188] LustreError: 432369:0:(niobuf.c:389:ptlrpc_register_bulk()) lustre-OST0001-osc-ffff988d04991000: LNetMEAttach failed x1818651264386177/1: rc = -12
[16578.347432] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[16578.361018] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[16578.544657] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[16578.595282] Lustre: Unmounted lustre-client
[16578.608509] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[16578.622060] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[16578.677976] Lustre: Mounted lustre-client
[16580.593418] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 415: lock revoke is not missing =========== 04:07:24 \(1734408444\)
[16580.971639] Lustre: DEBUG MARKER: == sanity test 415: lock revoke is not missing =========== 04:07:24 (1734408444)
[16581.671577] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16582.607456] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16594.092374] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16595.014841] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16596.243378] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 416: transaction start failure won\'t cause system hung ========================================================== 04:07:39 \(1734408459\)
[16596.614257] Lustre: DEBUG MARKER: == sanity test 416: transaction start failure won't cause system hung ========================================================== 04:07:39 (1734408459)
[16598.872788] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 417: disable remote dir, striped dir and dir migration ========================================================== 04:07:42 \(1734408462\)
[16599.247833] Lustre: DEBUG MARKER: == sanity test 417: disable remote dir, striped dir and dir migration ========================================================== 04:07:42 (1734408462)
[16599.676414] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_417 needs \>= 2 MDTs
[16600.043702] Lustre: DEBUG MARKER: SKIP: sanity test_417 needs >= 2 MDTs
[16600.510696] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 418: df and lfs df outputs match ========== 04:07:44 \(1734408464\)
[16600.881345] Lustre: DEBUG MARKER: == sanity test 418: df and lfs df outputs match ========== 04:07:44 (1734408464)
[16609.399652] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[16610.467095] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[16611.530393] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[16612.812732] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[16614.217274] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16616.213768] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16617.639918] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[16619.747335] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.*.lru_size=clear
[16621.163970] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16623.051086] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16625.763827] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 419: Verify open file by name doesn\'t crash kernel ========================================================== 04:08:09 \(1734408489\)
[16626.142833] Lustre: DEBUG MARKER: == sanity test 419: Verify open file by name doesn't crash kernel ========================================================== 04:08:09 (1734408489)
[16628.193952] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 420: clear SGID bit on non-directories for non-members ========================================================== 04:08:11 \(1734408491\)
[16628.741747] Lustre: DEBUG MARKER: == sanity test 420: clear SGID bit on non-directories for non-members ========================================================== 04:08:11 (1734408491)
[16630.820694] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421a: simple rm by fid ==================== 04:08:14 \(1734408494\)
[16631.190766] Lustre: DEBUG MARKER: == sanity test 421a: simple rm by fid ==================== 04:08:14 (1734408494)
[16633.236734] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421b: rm by fid on open file ============== 04:08:16 \(1734408496\)
[16633.604592] Lustre: DEBUG MARKER: == sanity test 421b: rm by fid on open file ============== 04:08:16 (1734408496)
[16635.588923] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421c: rm by fid against hardlinked files == 04:08:19 \(1734408499\)
[16635.958790] Lustre: DEBUG MARKER: == sanity test 421c: rm by fid against hardlinked files == 04:08:19 (1734408499)
[16636.689229] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16637.463417] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16640.319115] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421d: rmfid en masse ====================== 04:08:23 \(1734408503\)
[16640.691194] Lustre: DEBUG MARKER: == sanity test 421d: rmfid en masse ====================== 04:08:23 (1734408503)
[16641.399988] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16643.968799] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16653.227908] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421e: rmfid in DNE ======================== 04:08:36 \(1734408516\)
[16653.594348] Lustre: DEBUG MARKER: == sanity test 421e: rmfid in DNE ======================== 04:08:36 (1734408516)
[16654.019521] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_421e needs \>= 2 MDTs
[16654.417412] Lustre: DEBUG MARKER: SKIP: sanity test_421e needs >= 2 MDTs
[16654.879641] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421f: rmfid checks permissions ============ 04:08:38 \(1734408518\)
[16655.246557] Lustre: DEBUG MARKER: == sanity test 421f: rmfid checks permissions ============ 04:08:38 (1734408518)
[16655.520878] Lustre: DEBUG MARKER: mkdir -p /tmp/lustre-oKxJFg
[16655.533221] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,user_fid2path 10.240.22.189@tcp:/lustre /tmp/lustre-oKxJFg
[16655.583801] Lustre: Mounted lustre-client
[16657.168543] Lustre: DEBUG MARKER: grep -c /tmp/lustre-oKxJFg' ' /proc/mounts
[16657.182109] Lustre: DEBUG MARKER: lsof -t /tmp/lustre-oKxJFg
[16657.362450] Lustre: DEBUG MARKER: umount  /tmp/lustre-oKxJFg 2>&1
[16657.407246] Lustre: Unmounted lustre-client
[16657.861595] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421g: rmfid to return errors properly ===== 04:08:41 \(1734408521\)
[16658.229323] Lustre: DEBUG MARKER: == sanity test 421g: rmfid to return errors properly ===== 04:08:41 (1734408521)
[16658.657586] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_421g needs \>= 2 MDTs
[16659.025268] Lustre: DEBUG MARKER: SKIP: sanity test_421g needs >= 2 MDTs
[16659.486924] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 421h: rmfid with fileset mount ============ 04:08:42 \(1734408522\)
[16659.853758] Lustre: DEBUG MARKER: == sanity test 421h: rmfid with fileset mount ============ 04:08:42 (1734408522)
[16660.080040] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_other
[16660.092260] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre_other
[16660.137579] Lustre: Mounted lustre-client
[16660.145705] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[16660.160851] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre_other//d421h.sanity/subdir
[16660.179475] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre_other' ' /proc/mounts ||
[16660.179475] 			umount /mnt/lustre_other
[16660.222328] Lustre: Unmounted lustre-client
[16660.228331] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre/d421h.sanity/subdir /mnt/lustre_other
[16660.321672] Lustre: DEBUG MARKER: grep -c /mnt/lustre_other' ' /proc/mounts
[16660.334442] Lustre: DEBUG MARKER: lsof -t /mnt/lustre_other
[16660.516352] Lustre: DEBUG MARKER: umount  /mnt/lustre_other 2>&1
[16662.514326] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 422: kill a process with RPC in progress == 04:08:46 \(1734408526\)
[16662.881891] Lustre: DEBUG MARKER: == sanity test 422: kill a process with RPC in progress == 04:08:46 (1734408526)
[16662.961191] Lustre: DEBUG MARKER: lctl get_param -n at_max
[16663.314716] Lustre: DEBUG MARKER: lctl set_param at_max=0
[16677.853217] Autotest: Test running for 275 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[16684.102112] Lustre: 749525:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734408527/real 1734408527]  req@ffff988d026b09c0 x1818651270620544/t0(0) o101->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 576/1744 e 0 to 1 dl 1734408547 ref 2 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0
[16684.109905] Lustre: lustre-MDT0000-mdc-ffff988c14c64800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[16684.113041] Lustre: Skipped 3 previous similar messages
[16684.117429] Lustre: lustre-MDT0000-mdc-ffff988c14c64800: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[16704.581991] Lustre: 749525:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734408548/real 1734408548]  req@ffff988d026b09c0 x1818651270620544/t0(0) o101->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 576/1744 e 0 to 1 dl 1734408568 ref 2 fl Rpc:XQr/202/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0
[16704.582035] Lustre: lustre-MDT0000-mdc-ffff988c14c64800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[16704.587680] Lustre: 749525:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[16707.331333] Lustre: DEBUG MARKER: touch
[16725.061863] Lustre: 749525:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734408568/real 1734408568]  req@ffff988d026b09c0 x1818651270620544/t0(0) o101->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 576/1744 e 0 to 1 dl 1734408588 ref 2 fl Rpc:XQr/202/ffffffff rc 0/-1 job:'touch.0' uid:0 gid:0
[16725.070538] Lustre: lustre-MDT0000-mdc-ffff988c14c64800: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[16725.072517] Lustre: Skipped 1 previous similar message
[16725.118808] Lustre: DEBUG MARKER: lctl set_param at_max=600
[16727.968733] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 423: statfs should return a right data ==== 04:09:51 \(1734408591\)
[16728.339822] Lustre: DEBUG MARKER: == sanity test 423: statfs should return a right data ==== 04:09:51 (1734408591)
[16732.309359] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 424: simulate ENOMEM in ptl_send_rpc bulk reply ME attach ========================================================== 04:09:55 \(1734408595\)
[16732.681112] Lustre: DEBUG MARKER: == sanity test 424: simulate ENOMEM in ptl_send_rpc bulk reply ME attach ========================================================== 04:09:55 (1734408595)
[16732.775126] Lustre: *** cfs_fail_loc=522, val=0***
[16732.776118] LustreError: 432369:0:(niobuf.c:899:ptl_send_rpc()) LNetMEAttach failed: -12
[16734.810623] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 425: lock count should not exceed lru size ========================================================== 04:09:58 \(1734408598\)
[16735.182120] Lustre: DEBUG MARKER: == sanity test 425: lock count should not exceed lru size ========================================================== 04:09:58 (1734408598)
[16743.325868] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 426: splice test on Lustre ================ 04:10:06 \(1734408606\)
[16743.691132] Lustre: DEBUG MARKER: == sanity test 426: splice test on Lustre ================ 04:10:06 (1734408606)
[16746.109058] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 427: Failed DNE2 update request shouldn\'t corrupt updatelog ========================================================== 04:10:09 \(1734408609\)
[16746.482572] Lustre: DEBUG MARKER: == sanity test 427: Failed DNE2 update request shouldn't corrupt updatelog ========================================================== 04:10:09 (1734408609)
[16746.916081] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_427 needs \>= 2 MDTs
[16747.286932] Lustre: DEBUG MARKER: SKIP: sanity test_427 needs >= 2 MDTs
[16747.751199] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 428: large block size IO should not hang == 04:10:11 \(1734408611\)
[16748.117493] Lustre: DEBUG MARKER: == sanity test 428: large block size IO should not hang == 04:10:11 (1734408611)
[16839.877752] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 429: verify if opencache flag on client side does work ========================================================== 04:11:43 \(1734408703\)
[16840.250956] Lustre: DEBUG MARKER: == sanity test 429: verify if opencache flag on client side does work ========================================================== 04:11:43 (1734408703)
[16842.226287] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 430a: lseek: SEEK_DATA\/SEEK_HOLE basic functionality ========================================================== 04:11:45 \(1734408705\)
[16842.594312] Lustre: DEBUG MARKER: == sanity test 430a: lseek: SEEK_DATA/SEEK_HOLE basic functionality ========================================================== 04:11:45 (1734408705)
[16845.860479] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 430b: lseek: SEEK_DATA\/SEEK_HOLE special cases ========================================================== 04:11:49 \(1734408709\)
[16846.235609] Lustre: DEBUG MARKER: == sanity test 430b: lseek: SEEK_DATA/SEEK_HOLE special cases ========================================================== 04:11:49 (1734408709)
[16848.351978] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 430c: lseek: external tools check ========= 04:11:51 \(1734408711\)
[16848.727356] Lustre: DEBUG MARKER: == sanity test 430c: lseek: external tools check ========= 04:11:51 (1734408711)
[16850.742181] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 431: Restart transaction for IO =========== 04:11:54 \(1734408714\)
[16851.113367] Lustre: DEBUG MARKER: == sanity test 431: Restart transaction for IO =========== 04:11:54 (1734408714)
[16853.060615] bash (754640): drop_caches: 3
[16855.151696] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 432: mv dir from outside Lustre =========== 04:11:58 \(1734408718\)
[16855.521995] Lustre: DEBUG MARKER: == sanity test 432: mv dir from outside Lustre =========== 04:11:58 (1734408718)
[16866.939069] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 433: ldlm lock cancel releases dentries and inodes ========================================================== 04:12:10 \(1734408730\)
[16867.312796] Lustre: DEBUG MARKER: == sanity test 433: ldlm lock cancel releases dentries and inodes ========================================================== 04:12:10 (1734408730)
[16868.037437] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16868.805066] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16870.268571] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[16871.038077] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16875.770079] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 434: Client should not send RPCs for security.selinux with SElinux disabled ========================================================== 04:12:19 \(1734408739\)
[16876.143080] Lustre: DEBUG MARKER: == sanity test 434: Client should not send RPCs for security.selinux with SElinux disabled ========================================================== 04:12:19 (1734408739)
[16876.786201] Lustre: DEBUG MARKER: params=$(/usr/sbin/lctl get_param llite.*.xattr_cache);
[16876.786201] 			 [[ -z "" ]] && param= ||
[16876.786201] 			 param=$(grep  <<< "$params");
[16876.786201] 			 [[ -z $param ]] && param="$params";
[16876.786201] 			 while read s; do echo client $s;
[16876.786201] 			 done <<< "$param"
[16878.920841] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n llite.lustre-ffff988c14c64800.xattr_cache=1
[16879.383656] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 440: bash completion for lfs, lctl ======== 04:12:22 \(1734408742\)
[16879.754393] Lustre: DEBUG MARKER: == sanity test 440: bash completion for lfs, lctl ======== 04:12:22 (1734408742)
[16881.690877] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 442: truncate vs read\/write should not panic ========================================================== 04:12:25 \(1734408745\)
[16882.061398] Lustre: DEBUG MARKER: == sanity test 442: truncate vs read/write should not panic ========================================================== 04:12:25 (1734408745)
[16883.119438] LustreError: 758624:0:(llite_lib.c:3120:ll_truncate_inode_pages_final()) cfs_fail_timeout id 1430 sleeping for 5000ms
[16888.216894] LustreError: 758624:0:(llite_lib.c:3120:ll_truncate_inode_pages_final()) cfs_fail_timeout id 1430 awake
[16890.173103] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 460d: Check encrypt pools output ========== 04:12:33 \(1734408753\)
[16890.549208] Lustre: DEBUG MARKER: == sanity test 460d: Check encrypt pools output ========== 04:12:33 (1734408753)
[16892.639937] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600a: basic test for mlock\(\)ed file ======= 04:12:36 \(1734408756\)
[16893.012615] Lustre: DEBUG MARKER: == sanity test 600a: basic test for mlock()ed file ======= 04:12:36 (1734408756)
[16895.296099] bash (759398): drop_caches: 1
[16898.684704] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600b: mlock a file \(via vmtouch\) larger than max_cached_mb ========================================================== 04:12:42 \(1734408762\)
[16899.056005] Lustre: DEBUG MARKER: == sanity test 600b: mlock a file (via vmtouch) larger than max_cached_mb ========================================================== 04:12:42 (1734408762)
[16908.090040] bash (759908): drop_caches: 1
[16910.343137] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600c: Test I\/O when mlocked page count \> @max_cached_mb ========================================================== 04:12:53 \(1734408773\)
[16910.716166] Lustre: DEBUG MARKER: == sanity test 600c: Test I/O when mlocked page count > @max_cached_mb ========================================================== 04:12:53 (1734408773)
[16915.090533] bash (760406): drop_caches: 3
[16916.707297] bash (760406): drop_caches: 1
[16918.768573] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 600d: Test I\/O with limited LRU page slots \(some was mlocked\) ========================================================== 04:13:02 \(1734408782\)
[16919.142359] Lustre: DEBUG MARKER: == sanity test 600d: Test I/O with limited LRU page slots (some was mlocked) ========================================================== 04:13:02 (1734408782)
[16921.564312] bash (760923): drop_caches: 1
[16924.045611] bash (760923): drop_caches: 1
[16926.568054] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 801a: write barrier user interfaces and stat machine ========================================================== 04:13:10 \(1734408790\)
[16926.936054] Lustre: DEBUG MARKER: == sanity test 801a: write barrier user interfaces and stat machine ========================================================== 04:13:10 (1734408790)
[16927.677995] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 debug_mb=150
[16957.970960] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug_mb=11
[16958.756269] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16961.437744] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 801b: modification will be blocked by write barrier ========================================================== 04:13:44 \(1734408824\)
[16961.805768] Lustre: DEBUG MARKER: == sanity test 801b: modification will be blocked by write barrier ========================================================== 04:13:44 (1734408824)
[16962.543849] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug=-1 debug_mb=150
[16963.301466] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[16965.989586] Lustre: 763017:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS  req@ffff988d026b3400 x1818651274642944/t0(0) o36->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 488/456 e 0 to 0 dl 1734408846 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'rm.0' uid:0 gid:0
[16965.994277] Lustre: 763017:0:(client.c:1532:after_reply()) Skipped 2 previous similar messages
[16971.665659] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param debug_mb=11
[16973.945217] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 801c: rescan barrier bitmap =============== 04:13:57 \(1734408837\)
[16974.319036] Lustre: DEBUG MARKER: == sanity test 801c: rescan barrier bitmap =============== 04:13:57 (1734408837)
[16974.748822] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_801c needs \>= 2 MDTs
[16975.116048] Lustre: DEBUG MARKER: SKIP: sanity test_801c needs >= 2 MDTs
[16975.583212] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 802b: be able to set MDTs to readonly ===== 04:13:59 \(1734408839\)
[16975.951627] Lustre: DEBUG MARKER: == sanity test 802b: be able to set MDTs to readonly ===== 04:13:59 (1734408839)
[16978.885677] Autotest: Test running for 280 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[16980.161525] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 802c: be able to set OFDs to readonly ===== 04:14:03 \(1734408843\)
[16980.532219] Lustre: DEBUG MARKER: == sanity test 802c: be able to set OFDs to readonly ===== 04:14:03 (1734408843)
[16984.455026] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 803a: verify agent object for remote object ========================================================== 04:14:07 \(1734408847\)
[16984.824565] Lustre: DEBUG MARKER: == sanity test 803a: verify agent object for remote object ========================================================== 04:14:07 (1734408847)
[16985.255325] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_803a needs \>= 2 MDTs
[16985.623002] Lustre: DEBUG MARKER: SKIP: sanity test_803a needs >= 2 MDTs
[16986.123791] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 803b: remote object can getattr from cache ========================================================== 04:14:09 \(1734408849\)
[16986.496392] Lustre: DEBUG MARKER: == sanity test 803b: remote object can getattr from cache ========================================================== 04:14:09 (1734408849)
[16986.925138] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_803b needs \>= 2 MDTs
[16987.296843] Lustre: DEBUG MARKER: SKIP: sanity test_803b needs >= 2 MDTs
[16987.764302] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 804: verify agent entry for remote entry == 04:14:11 \(1734408851\)
[16988.135769] Lustre: DEBUG MARKER: == sanity test 804: verify agent entry for remote entry == 04:14:11 (1734408851)
[16988.564682] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_804 needs \>= 2 MDTs
[16988.931146] Lustre: DEBUG MARKER: SKIP: sanity test_804 needs >= 2 MDTs
[16989.400234] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 805: ZFS can remove from full fs ========== 04:14:12 \(1734408852\)
[16989.771401] Lustre: DEBUG MARKER: == sanity test 805: ZFS can remove from full fs ========== 04:14:12 (1734408852)
[16990.525668] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_805 ZFS specific test
[16990.900113] Lustre: DEBUG MARKER: SKIP: sanity test_805 ZFS specific test
[16991.373013] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 806: Verify Lazy Size on MDS ============== 04:14:14 \(1734408854\)
[16991.745494] Lustre: DEBUG MARKER: == sanity test 806: Verify Lazy Size on MDS ============== 04:14:14 (1734408854)
[16997.765809] Lustre: DEBUG MARKER: multiop /mnt/lustre/f806.sanity Oz0w1048576c
[16998.606277] Lustre: DEBUG MARKER: multiop /mnt/lustre/f806.sanity Oz1048576w1048576c
[17003.885722] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 807a: verify LSOM syncing tool ============ 04:14:27 \(1734408867\)
[17004.260146] Lustre: DEBUG MARKER: == sanity test 807a: verify LSOM syncing tool ============ 04:14:27 (1734408867)
[17005.926810] Lustre: DEBUG MARKER: multiop /mnt/lustre/f807a.sanity Oz0w1048576c
[17007.075211] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17008.345030] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks osc
[17008.788819] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks osc
[17009.464599] Lustre: DEBUG MARKER: sync ; sleep 5 ; sync
[17018.912276] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 807b: verify lfs somsync utility ========== 04:14:42 \(1734408882\)
[17019.284423] Lustre: DEBUG MARKER: == sanity test 807b: verify lfs somsync utility ========== 04:14:42 (1734408882)
[17019.574160] Lustre: DEBUG MARKER: multiop /mnt/lustre/f807b.sanity Oz0w1048576c
[17020.684120] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17021.941664] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks osc
[17022.452897] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks osc
[17023.067410] Lustre: DEBUG MARKER: sync ; sleep 5 ; sync
[17030.995981] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 808: Check trusted.som xattr not logged in Changelogs ========================================================== 04:14:54 \(1734408894\)
[17031.367067] Lustre: DEBUG MARKER: == sanity test 808: Check trusted.som xattr not logged in Changelogs ========================================================== 04:14:54 (1734408894)
[17035.632881] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 809: Verify no SOM xattr store for DoM-only files ========================================================== 04:14:59 \(1734408899\)
[17036.002475] Lustre: DEBUG MARKER: == sanity test 809: Verify no SOM xattr store for DoM-only files ========================================================== 04:14:59 (1734408899)
[17038.027295] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 810: partial page writes on ZFS \(LU-11663\) ========================================================== 04:15:01 \(1734408901\)
[17038.396686] Lustre: DEBUG MARKER: == sanity test 810: partial page writes on ZFS (LU-11663) ========================================================== 04:15:01 (1734408901)
[17038.438270] Lustre: *** cfs_fail_loc=411, val=0***
[17038.439247] Lustre: Skipped 1 previous similar message
[17038.993781] Lustre: *** cfs_fail_loc=411, val=0***
[17038.994821] Lustre: Skipped 5 previous similar messages
[17040.080149] Lustre: *** cfs_fail_loc=411, val=0***
[17040.081242] Lustre: Skipped 13 previous similar messages
[17042.126501] Lustre: *** cfs_fail_loc=411, val=0***
[17042.127589] Lustre: Skipped 31 previous similar messages
[17044.225990] Lustre: DEBUG MARKER: /usr/sbin/lctl mark set checksum type to t10ip4K, rc = 0
[17044.594640] Lustre: DEBUG MARKER: set checksum type to t10ip4K, rc = 0
[17045.055043] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 812a: do not drop reqs generated when imp is going to idle \(LU-11951\) ========================================================== 04:15:08 \(1734408908\)
[17045.427284] Lustre: DEBUG MARKER: == sanity test 812a: do not drop reqs generated when imp is going to idle (LU-11951) ========================================================== 04:15:08 (1734408908)
[17045.477793] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17045.509514] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17046.321304] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17046.679225] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17047.083649] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17047.436677] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17047.783999] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17047.816998] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17048.637460] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17048.992546] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17058.498584] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in CONNECTING state after 9 sec
[17058.849109] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in CONNECTING state after 9 sec
[17061.127400] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 812b: do not drop no resend request for idle connect ========================================================== 04:15:24 \(1734408924\)
[17061.500640] Lustre: DEBUG MARKER: == sanity test 812b: do not drop no resend request for idle connect ========================================================== 04:15:24 (1734408924)
[17061.548798] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17061.580222] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17062.400382] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17062.751415] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17063.153886] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17063.503394] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17063.850686] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17063.883406] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17064.704331] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17065.056784] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state CONNECTING osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17074.562482] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in CONNECTING state after 9 sec
[17074.925885] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in CONNECTING state after 9 sec
[17075.329758] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17075.363485] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17076.196550] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17076.548384] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17089.092115] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in IDLE state after 12 sec
[17089.535658] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in IDLE state after 12 sec
[17091.590177] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 812c: idle import vs lock enqueue race ==== 04:15:55 \(1734408955\)
[17091.964865] Lustre: DEBUG MARKER: == sanity test 812c: idle import vs lock enqueue race ==== 04:15:55 (1734408955)
[17092.022097] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17092.054884] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17092.882710] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17093.236010] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17093.638524] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17093.990971] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17103.939759] LustreError: 725304:0:(import.c:2047:ptlrpc_disconnect_and_idle_import()) cfs_race id 533 sleeping
[17109.023160] LustreError: 773615:0:(osc_lock.c:1017:osc_lock_enqueue()) cfs_fail_race id 533 waking
[17109.024919] LustreError: 725304:0:(import.c:2047:ptlrpc_disconnect_and_idle_import()) cfs_fail_race id 533 awake: rc=1
[17109.540988] LustreError: 773615:0:(osc_lock.c:1017:osc_lock_enqueue()) cfs_fail_race id 533 waking
[17112.067156] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 813: File heat verfication ================ 04:16:15 \(1734408975\)
[17112.436853] Lustre: DEBUG MARKER: == sanity test 813: File heat verfication ================ 04:16:15 (1734408975)
[17241.003520] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 814: sparse cp works as expected \(LU-12361\) ========================================================== 04:18:24 \(1734409104\)
[17241.377270] Lustre: DEBUG MARKER: == sanity test 814: sparse cp works as expected (LU-12361) ========================================================== 04:18:24 (1734409104)
[17243.834454] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 815: zero byte tiny write doesn\'t hang \(LU-12382\) ========================================================== 04:18:27 \(1734409107\)
[17244.208464] Lustre: DEBUG MARKER: == sanity test 815: zero byte tiny write doesn't hang (LU-12382) ========================================================== 04:18:27 (1734409107)
[17246.151708] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 816: do not reset lru_resize on idle reconnect ========================================================== 04:18:29 \(1734409109\)
[17246.520174] Lustre: DEBUG MARKER: == sanity test 816: do not reset lru_resize on idle reconnect ========================================================== 04:18:29 (1734409109)
[17246.577435] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17246.609367] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17247.434161] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17247.784179] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state FULL osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17248.186920] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17248.538034] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in FULL state after 0 sec
[17248.889477] Lustre: DEBUG MARKER: lctl get_param -n at_min
[17248.921972] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17249.749814] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17250.103199] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state IDLE osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid 50
[17263.655954] Lustre: DEBUG MARKER: /usr/sbin/lctl mark osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in IDLE state after 13 sec
[17264.090705] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff988c14c64800.ost_server_uuid in IDLE state after 13 sec
[17266.437852] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_817 skipping ALWAYS excluded test 817
[17266.811988] Lustre: DEBUG MARKER: SKIP: sanity test_817 skipping ALWAYS excluded test 817
[17267.274022] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 818: unlink with failed llog ============== 04:18:50 \(1734409130\)
[17267.643477] Lustre: DEBUG MARKER: == sanity test 818: unlink with failed llog ============== 04:18:50 (1734409130)
[17272.899544] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[17272.903461] Lustre: lustre-MDT0000-mdc-ffff988c14c64800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[17272.903954] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f18b71fb to 0xdf05ec28f19623f7
[17272.906515] Lustre: Skipped 1 previous similar message
[17272.909012] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[17272.931661] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988d0134e700 x1818651274768512/t68719521411(68719521411) o101->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 576/608 e 0 to 0 dl 1734409152 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'md5sum.0' uid:0 gid:0
[17272.936806] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 2 previous similar messages
[17279.618744] Autotest: Test running for 285 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[17293.390175] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[17293.394876] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f19623f7 to 0xdf05ec28f19627e0
[17293.427109] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988d0134e700 x1818651274768512/t68719521411(68719521411) o101->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 576/608 e 0 to 0 dl 1734409173 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'md5sum.0' uid:0 gid:0
[17293.432284] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 4 previous similar messages
[17296.395231] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17297.674539] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[17298.201117] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[17298.415700] Lustre: DEBUG MARKER: lctl get_param -n at_max
[17298.814911] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[17299.267591] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[17299.302495] Lustre: 432371:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734409147/real 1734409147]  req@ffff988d026b0000 x1818651274962944/t0(0) o400->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734409163 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[17301.356570] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 819a: too big niobuf in read ============== 04:19:24 \(1734409164\)
[17301.725447] Lustre: DEBUG MARKER: == sanity test 819a: too big niobuf in read ============== 04:19:24 (1734409164)
[17304.044362] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 819b: too big niobuf in write ============= 04:19:27 \(1734409167\)
[17304.413870] Lustre: DEBUG MARKER: == sanity test 819b: too big niobuf in write ============= 04:19:27 (1734409167)
[17307.702442] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 820: update max EA from open intent ======= 04:19:31 \(1734409171\)
[17308.081185] Lustre: DEBUG MARKER: == sanity test 820: update max EA from open intent ======= 04:19:31 (1734409171)
[17308.510963] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_820 needs \>= 2 MDTs
[17308.885390] Lustre: DEBUG MARKER: SKIP: sanity test_820 needs >= 2 MDTs
[17309.360084] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 823: Setting create_count \> OST_MAX_PRECREATE is lowered to maximum ========================================================== 04:19:32 \(1734409172\)
[17309.733935] Lustre: DEBUG MARKER: == sanity test 823: Setting create_count > OST_MAX_PRECREATE is lowered to maximum ========================================================== 04:19:32 (1734409172)
[17312.104513] Lustre: DEBUG MARKER: /usr/sbin/lctl mark setting create_count to 100200:
[17312.475626] Lustre: DEBUG MARKER: setting create_count to 100200:
[17312.905053] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  -result- count: 9984 with max: 20000, expecting: 9984
[17313.274830] Lustre: DEBUG MARKER: -result- count: 9984 with max: 20000, expecting: 9984
[17317.540649] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 831: throttling unlink\/setattr queuing on OSP ========================================================== 04:19:41 \(1734409181\)
[17317.910621] Lustre: DEBUG MARKER: == sanity test 831: throttling unlink/setattr queuing on OSP ========================================================== 04:19:41 (1734409181)
[17319.943343] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[17321.091841] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[17322.578431] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=0
[17325.029114] Lustre: 780806:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS  req@ffff988d019e0340 x1818651275519488/t0(0) o36->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 488/456 e 0 to 0 dl 1734409204 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[17331.303232] Lustre: 780806:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS  req@ffff988d11cd2d80 x1818651275546368/t0(0) o36->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 488/456 e 0 to 0 dl 1734409211 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[17337.410757] Lustre: 780806:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS  req@ffff988d02035040 x1818651275554304/t0(0) o36->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 488/456 e 0 to 0 dl 1734409217 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[17343.522686] Lustre: 780806:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS  req@ffff988d020356c0 x1818651275581568/t0(0) o36->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 488/456 e 0 to 0 dl 1734409223 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[17343.527570] Lustre: 780806:0:(client.c:1532:after_reply()) Skipped 1 previous similar message
[17355.846623] Lustre: 780806:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS  req@ffff988d17c58000 x1818651275616640/t0(0) o36->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 488/456 e 0 to 0 dl 1734409235 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[17355.851410] Lustre: 780806:0:(client.c:1532:after_reply()) Skipped 1 previous similar message
[17375.298596] Lustre: 780806:0:(client.c:1532:after_reply()) @@@ resending request on EINPROGRESS  req@ffff988d13cf9040 x1818651275697792/t0(0) o36->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 488/456 e 0 to 0 dl 1734409255 ref 2 fl Rpc:RQU/202/0 rc 0/-115 job:'unlinkmany.0' uid:0 gid:0
[17375.303416] Lustre: 780806:0:(client.c:1532:after_reply()) Skipped 2 previous similar messages
[17405.294276] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -n debug=trace+inode+super+iotrace+malloc+cache+info+ioctl+neterror+net+warning+buffs+other+dentry+nettrace+page+dlmtrace+error+emerg+ha+rpctrace+vfstrace+reada+mmap+config+console+quota+sec+lfsck+hsm+snapshot+layout
[17412.995462] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 832: lfs rm_entry ========================= 04:21:16 \(1734409276\)
[17413.366085] Lustre: DEBUG MARKER: == sanity test 832: lfs rm_entry ========================= 04:21:16 (1734409276)
[17413.791485] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_832 needs \>= 2 MDTs
[17414.159084] Lustre: DEBUG MARKER: SKIP: sanity test_832 needs >= 2 MDTs
[17414.623767] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 833: Mixed buffered\/direct read and write should not return -EIO ========================================================== 04:21:18 \(1734409278\)
[17414.989867] Lustre: DEBUG MARKER: == sanity test 833: Mixed buffered/direct read and write should not return -EIO ========================================================== 04:21:18 (1734409278)
[17471.379575] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_842 skipping SLOW test 842
[17471.762306] Lustre: DEBUG MARKER: SKIP: sanity test_842 skipping SLOW test 842
[17472.241391] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 850: lljobstat can parse living and aggregated job_stats ========================================================== 04:22:15 \(1734409335\)
[17472.634698] Lustre: DEBUG MARKER: == sanity test 850: lljobstat can parse living and aggregated job_stats ========================================================== 04:22:15 (1734409335)
[17474.764263] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 851: fanotify can monitor open\/read\/write\/close events for lustre fs ========================================================== 04:22:18 \(1734409338\)
[17475.136798] Lustre: DEBUG MARKER: == sanity test 851: fanotify can monitor open/read/write/close events for lustre fs ========================================================== 04:22:18 (1734409338)
[17476.494370] Lustre: DEBUG MARKER: stat -c %s /tmp/report_test_851_15641
[17477.148427] Lustre: DEBUG MARKER: stat -c %s /tmp/report_test_851_15641
[17479.412609] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 852: mkdir using intent lock for striped directory ========================================================== 04:22:22 \(1734409342\)
[17479.782638] Lustre: DEBUG MARKER: == sanity test 852: mkdir using intent lock for striped directory ========================================================== 04:22:22 (1734409342)
[17480.214176] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity test_852 needs \>= 2 MDTs
[17480.586264] Lustre: DEBUG MARKER: SKIP: sanity test_852 needs >= 2 MDTs
[17481.055782] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 900: umount should not race with any mgc requeue thread ========================================================== 04:22:24 \(1734409344\)
[17481.426678] Lustre: DEBUG MARKER: == sanity test 900: umount should not race with any mgc requeue thread ========================================================== 04:22:24 (1734409344)
[17498.177331] Lustre: 432370:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1734409346/real 1734409346]  req@ffff988d0134f0c0 x1818651282040192/t0(0) o400->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734409362 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0
[17498.178496] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[17498.182938] Lustre: 432370:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[17498.182980] Lustre: lustre-MDT0000-mdc-ffff988c14c64800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[17498.190126] Lustre: Skipped 1 previous similar message
[17498.191486] Lustre: Evicted from MGS (at 10.240.22.189@tcp) after server handle changed from 0xdf05ec28f19627e0 to 0xdf05ec28f1972997
[17498.194240] Lustre: MGC10.240.22.189@tcp: Connection restored to 10.240.22.189@tcp (at 10.240.22.189@tcp)
[17498.195986] Lustre: Skipped 3 previous similar messages
[17498.212969] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff988d0134e700 x1818651274768512/t68719521411(68719521411) o101->lustre-MDT0000-mdc-ffff988c14c64800@10.240.22.189@tcp:12/10 lens 576/608 e 0 to 0 dl 1734409378 ref 2 fl Interpret:RPQU/204/0 rc 301/301 job:'md5sum.0' uid:0 gid:0
[17498.218069] LustreError: 432368:0:(client.c:3284:ptlrpc_replay_interpret()) Skipped 4 previous similar messages
[17499.905312] LustreError: 739292:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 sleeping for 20000ms
[17502.487403] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[17503.771596] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount \(FULL\|IDLE\) mdc.lustre-MDT0000-mdc-\*.mds_server_uuid
[17504.294687] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid
[17504.447383] Lustre: DEBUG MARKER: lctl get_param -n at_max
[17504.846182] Lustre: DEBUG MARKER: /usr/sbin/lctl mark mdc.lustre-MDT0000-mdc-\*.mds_server_uuid in FULL state after 0 sec
[17505.270530] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec
[17507.254311] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[17507.254311] if [ $running -ne 0 ] ; then
[17507.254311] echo Stopping client $(hostname) /mnt/lustre opts:;
[17507.254311] lsof /mnt/lustre || need_kill=no;
[17507.254311] if [ x != x -a x$need_kill != xno ]; then
[17507.254311]     pids=$(lsof -t /mnt/lustre | sort -u);
[17507.254311]     if 
[17519.993178] LustreError: 739292:0:(mgc_request.c:1762:mgc_process_log()) cfs_fail_timeout id 903 awake
[17520.031150] Lustre: Unmounted lustre-client
[17520.032078] Lustre: Skipped 1 previous similar message
[17520.366147] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[17520.366147] if [ $running -ne 0 ] ; then
[17520.366147] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[17520.366147] lsof /mnt/lustre2 || need_kill=no;
[17520.366147] if [ x != x -a x$need_kill != xno ]; then
[17520.366147]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[17520.366147]    
[17580.489798] Autotest: Test running for 290 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[17615.003428] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1 || true
[17617.381773] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[17617.381773] lctl dl | grep ' ST ' || true
[17617.784567] Key type lgssc unregistered
[17617.848811] LNet: 786484:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[17618.896778] LNet: Removed LNI 10.240.22.180@tcp
[17619.129849] Key type .llcrypt unregistered
[17619.130800] Key type ._llcrypt unregistered
[17620.726922] Key type ._llcrypt registered
[17620.727871] Key type .llcrypt registered
[17620.969381] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[17621.791802] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[17621.848898] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[17621.850132] LNet: Accept all, port 7988
[17623.388587] Key type lgssc registered
[17623.607764] Lustre: Echo OBD driver; http://www.lustre.org/
[17627.522272] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[17627.522272] mpts=$(mount | grep -c /mnt/lustre' ');
[17627.522272] if [ $running -ne $mpts ]; then
[17627.522272]     echo $(hostname) env are INSANE!;
[17627.522272]     exit 1;
[17627.522272] fi
[17628.312940] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[17628.312940] mpts=$(mount | grep -c /mnt/lustre2' ');
[17628.312940] if [ $running -ne $mpts ]; then
[17628.312940]     echo $(hostname) env are INSANE!;
[17628.312940]     exit 1;
[17628.312940] fi
[17692.337555] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[17692.353089] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[17693.401919] Lustre: Mounted lustre-client
[17693.750173] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[17694.451496] Lustre: DEBUG MARKER: 
[17694.451496] running=$(mount | grep -c /mnt/lustre' ');
[17694.451496] rc=0;
[17694.451496] if [ $running -eq 0 ] ; then
[17694.451496] 	mkdir -p /mnt/lustre;
[17694.451496] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[17694.451496] 	rc=$?;
[17694.451496] fi;
[17694.451496] exit $rc
[17695.212818] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[17700.004211] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[17700.373439] Lustre: DEBUG MARKER: Using TIMEOUT=20
[17700.799173] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[17701.509974] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[17708.809115] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 901: don\'t leak a mgc lock on client umount ========================================================== 04:26:12 \(1734409572\)
[17709.182123] Lustre: DEBUG MARKER: == sanity test 901: don't leak a mgc lock on client umount ========================================================== 04:26:12 (1734409572)
[17710.215503] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[17710.229274] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[17710.406913] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[17710.451976] Lustre: Unmounted lustre-client
[17710.464067] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[17710.476848] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[17710.534302] Lustre: Mounted lustre-client
[17712.498180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 902: test short write doesn\'t hang lustre ========================================================== 04:26:16 \(1734409576\)
[17712.872243] Lustre: DEBUG MARKER: == sanity test 902: test short write doesn't hang lustre ========================================================== 04:26:16 (1734409576)
[17712.919117] Lustre: *** cfs_fail_loc=2001415, val=0***
[17714.955843] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 903: Test long page discard does not cause evictions ========================================================== 04:26:18 \(1734409578\)
[17715.325974] Lustre: DEBUG MARKER: == sanity test 903: Test long page discard does not cause evictions ========================================================== 04:26:18 (1734409578)
[17720.951545] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[17741.039887] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[17741.049896] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[17761.143773] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[17761.153919] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[17781.163660] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[17781.173807] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[17801.267534] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[17801.280361] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[17821.367412] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[17821.380144] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 sleeping for 20000ms
[17841.463332] LustreError: 790579:0:(osc_cache.c:3082:osc_page_gang_lookup()) cfs_fail_timeout id 417 awake
[17854.822904] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 904: virtual project ID xattr ============= 04:28:38 \(1734409718\)
[17855.191561] Lustre: DEBUG MARKER: == sanity test 904: virtual project ID xattr ============= 04:28:38 (1734409718)
[17859.606807] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 905: bad or new opcode should not stuck client ========================================================== 04:28:43 \(1734409723\)
[17859.976841] Lustre: DEBUG MARKER: == sanity test 905: bad or new opcode should not stuck client ========================================================== 04:28:43 (1734409723)
[17860.402309] LustreError: lustre-OST0001-osc-ffff988d39e87800: operation ost_ladvise to node 10.240.22.182@tcp failed: rc = -95
[17862.324461] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 906: Simple test for io_uring I\/O engine via fio ========================================================== 04:28:45 \(1734409725\)
[17862.697241] Lustre: DEBUG MARKER: == sanity test 906: Simple test for io_uring I/O engine via fio ========================================================== 04:28:45 (1734409725)
[17881.180944] Autotest: Test running for 295 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[18171.838187] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test 907: write rpc error during unlink ======== 04:33:55 \(1734410035\)
[18172.215220] Lustre: DEBUG MARKER: == sanity test 907: write rpc error during unlink ======== 04:33:55 (1734410035)
[18173.813846] LustreError: lustre-OST0000-osc-ffff988d39e87800: operation ost_write to node 10.240.22.182@tcp failed: rc = -3
[18173.815984] LustreError: Skipped 13 previous similar messages
[18175.746466] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity test complete, duration 17427 sec ============== 04:33:59 \(1734410039\)
[18176.117329] Lustre: DEBUG MARKER: == sanity test complete, duration 17427 sec ============== 04:33:59 (1734410039)
[18176.576385] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: start cleanup 04:34:00 \(1734410040\) ===
[18176.946032] Lustre: DEBUG MARKER: === sanity: start cleanup 04:34:00 (1734410040) ===
[18182.070169] Autotest: Test running for 300 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[18211.657405] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity: finish cleanup 04:34:35 \(1734410075\) ===
[18212.033993] Lustre: DEBUG MARKER: === sanity: finish cleanup 04:34:35 (1734410075) ===
[18212.465900] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: sanity-lnet ============----- Tue Dec 17 04:34:36 UTC 2024
[18212.827435] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-lnet ============----- Tue Dec 17 04:34:36 UTC 2024
[18214.295471] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[18217.191841] Lustre: DEBUG MARKER: cat /etc/system-release
[18217.203878] Lustre: DEBUG MARKER: test -r /etc/os-release
[18217.214903] Lustre: DEBUG MARKER: cat /etc/os-release
[18217.602155] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[18217.602155] if [ $running -ne 0 ] ; then
[18217.602155] echo Stopping client $(hostname) /mnt/lustre opts:;
[18217.602155] lsof /mnt/lustre || need_kill=no;
[18217.602155] if [ x != x -a x$need_kill != xno ]; then
[18217.602155]     pids=$(lsof -t /mnt/lustre | sort -u);
[18217.602155]     if 
[18242.790869] Lustre: Unmounted lustre-client
[18243.121751] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[18243.121751] if [ $running -ne 0 ] ; then
[18243.121751] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[18243.121751] lsof /mnt/lustre2 || need_kill=no;
[18243.121751] if [ x != x -a x$need_kill != xno ]; then
[18243.121751]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[18243.121751]    
[18280.554997] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18280.554997] lctl dl | grep ' ST ' || true
[18280.949135] Key type lgssc unregistered
[18281.024817] LNet: 800823:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18281.048827] LNet: Removed LNI 10.240.22.180@tcp
[18281.265553] Key type .llcrypt unregistered
[18281.266495] Key type ._llcrypt unregistered
[18293.475166] Key type ._llcrypt registered
[18293.476119] Key type .llcrypt registered
[18293.546649] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18294.315854] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18294.320367] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18294.321640] LNet: Accept all, port 7988
[18294.324738] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show
[18297.009620] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests: 253 254
[18297.444733] Lustre: DEBUG MARKER: excepting tests: 253 254
[18297.460149] LNet: 801899:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18297.472677] LNet: Removed LNI 10.240.22.180@tcp
[18297.479612] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18297.479612] lctl dl | grep ' ST ' || true
[18297.604669] Key type .llcrypt unregistered
[18297.607298] Key type ._llcrypt unregistered
[18297.861313] Key type ._llcrypt registered
[18297.862306] Key type .llcrypt registered
[18298.288029] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 04:36:01 \(1734410161\)
[18298.645174] Lustre: DEBUG MARKER: == sanity-lnet test 0: Export empty config, import the config, compare ========================================================== 04:36:01 (1734410161)
[18298.693254] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18299.458231] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18299.471623] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18299.474935] LNet: 802174:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18299.479081] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18301.273987] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18301.278132] LNet: 802182:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18303.075437] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 1: Add peer with single nid \(tcp\) ==== 04:36:06 \(1734410166\)
[18303.429333] Lustre: DEBUG MARKER: == sanity-lnet test 1: Add peer with single nid (tcp) ==== 04:36:06 (1734410166)
[18303.444962] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18303.448294] LNet: 802632:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18303.452779] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18303.464026] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp
[18305.249452] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 2: Add peer with single nid \(o2ib\) === 04:36:08 \(1734410168\)
[18305.604696] Lustre: DEBUG MARKER: == sanity-lnet test 2: Add peer with single nid (o2ib) === 04:36:08 (1734410168)
[18305.621832] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18305.627795] LNet: 803090:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18305.632126] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18305.643934] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 2.2.2.2@o2ib
[18307.414141] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 04:36:10 \(1734410170\)
[18307.766128] Lustre: DEBUG MARKER: == sanity-lnet test 3: Add peer with tcp primary o2ib secondary ========================================================== 04:36:10 (1734410170)
[18307.785830] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18307.789261] LNet: 803547:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18307.793744] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18307.804938] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 3.3.3.3@tcp --nid 3.3.3.3@o2ib
[18309.588616] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 4: Add peer with nidrange \(tcp\) ====== 04:36:13 \(1734410173\)
[18309.940420] Lustre: DEBUG MARKER: == sanity-lnet test 4: Add peer with nidrange (tcp) ====== 04:36:13 (1734410173)
[18309.955877] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18309.961707] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18309.973269] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-3]@tcp
[18310.036608] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 4.4.4.4@tcp --nid 4.4.4.[1-4]@tcp
[18311.807113] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 5: Add peer with nidrange \(o2ib\) ===== 04:36:15 \(1734410175\)
[18312.160381] Lustre: DEBUG MARKER: == sanity-lnet test 5: Add peer with nidrange (o2ib) ===== 04:36:15 (1734410175)
[18312.174473] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18312.179425] LNet: 804467:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18312.181107] LNet: 804467:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 1 previous similar message
[18312.185360] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18312.196805] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib
[18312.261319] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1-4]@o2ib
[18312.324899] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 5.5.5.5@o2ib --nid 5.5.5.[1,2,3-4]@o2ib
[18314.099408] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 6: Add peer with multiple nidranges == 04:36:17 \(1734410177\)
[18314.451763] Lustre: DEBUG MARKER: == sanity-lnet test 6: Add peer with multiple nidranges == 04:36:17 (1734410177)
[18314.468592] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18314.474422] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18314.486079] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 6.6.6.6@tcp --nid 6.6.[6-7].[0-4/2]@tcp,6.6.[1-4/2].[0-6/3]@o2ib,[6-12/4]@gni,[6-12/4]@kfi
[18316.260887] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 7: Various peer delete tests ========= 04:36:19 \(1734410179\)
[18316.615299] Lustre: DEBUG MARKER: == sanity-lnet test 7: Various peer delete tests ========= 04:36:19 (1734410179)
[18316.630107] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18316.636051] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18316.646126] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@tcp
[18316.651720] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@tcp
[18316.712005] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@o2ib
[18316.717737] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@o2ib
[18316.776944] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[8-12]@tcp
[18316.782995] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@tcp
[18316.842170] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@o2ib --nid 7.7.7.[8-12]@o2ib
[18316.848233] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@o2ib
[18316.907374] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7.7.7.7@tcp --nid 7.7.7.[9-12]@tcp,7.7.7.[13-15]@o2ib
[18316.913720] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7.7.7.7@tcp
[18316.972952] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@gni
[18316.978304] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@gni
[18317.036929] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni
[18317.042757] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@gni
[18317.101820] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@kfi
[18317.107337] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@kfi
[18317.165819] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@kfi --nid [8-12]@kfi
[18317.171653] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@kfi
[18317.230289] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 7@gni --nid [8-12]@gni,7.7.7.[1-4]@tcp,7.7.7.[5-9]@o2ib,[1-5]@kfi
[18317.236767] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 7@gni
[18317.295411] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp
[18317.301766] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@tcp --nid 1.1.1.1@tcp,2.2.2.2@tcp,3.3.3.3@tcp,4.4.4.4@tcp
[18319.078438] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 8: Delete single secondary nid from peer \(tcp\) ========================================================== 04:36:22 \(1734410182\)
[18319.430794] Lustre: DEBUG MARKER: == sanity-lnet test 8: Delete single secondary nid from peer (tcp) ========================================================== 04:36:22 (1734410182)
[18319.449639] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18319.455500] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18319.467007] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 8.8.8.8@tcp --nid 8.8.8.[10-15]@tcp
[18319.473136] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 8.8.8.8@tcp --nid 8.8.8.13@tcp
[18321.243307] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 9: Delete all secondary nids from peer \(tcp\) ========================================================== 04:36:24 \(1734410184\)
[18321.596061] Lustre: DEBUG MARKER: == sanity-lnet test 9: Delete all secondary nids from peer (tcp) ========================================================== 04:36:24 (1734410184)
[18321.611832] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18321.615165] LNet: 806390:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18321.616846] LNet: 806390:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 3 previous similar messages
[18321.621431] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18321.632920] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp
[18321.639128] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 9.9.9.9@tcp --nid 9.9.9.[11-16]@tcp
[18323.419446] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 10: Delete single secondary nid from peer \(o2ib\) ========================================================== 04:36:26 \(1734410186\)
[18323.779018] Lustre: DEBUG MARKER: == sanity-lnet test 10: Delete single secondary nid from peer (o2ib) ========================================================== 04:36:26 (1734410186)
[18323.795474] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18323.801389] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18323.812914] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 10.10.10.10@tcp --nid 10.10.10.[12-16]@tcp
[18323.819080] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 10.10.10.10@tcp --nid 10.10.10.14@tcp
[18325.599791] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 11: Delete all secondary nids from peer \(o2ib\) ========================================================== 04:36:29 \(1734410189\)
[18325.955995] Lustre: DEBUG MARKER: == sanity-lnet test 11: Delete all secondary nids from peer (o2ib) ========================================================== 04:36:29 (1734410189)
[18325.973581] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18325.979566] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18325.991104] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp
[18325.997587] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 11.11.11.11@tcp --nid 11.11.11.[13-17]@tcp
[18327.816308] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 12: Delete a secondary nid from peer \(tcp and o2ib\) ========================================================== 04:36:31 \(1734410191\)
[18328.170171] Lustre: DEBUG MARKER: == sanity-lnet test 12: Delete a secondary nid from peer (tcp and o2ib) ========================================================== 04:36:31 (1734410191)
[18328.186376] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18328.192336] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18328.203587] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 12.12.12.12@o2ib --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp
[18328.210743] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 12.12.12.12@o2ib --nid 13.15.13.13@o2ib,15.17.1.15@tcp
[18329.996478] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 13: Delete all secondary nids from peer \(tcp and o2ib\) ========================================================== 04:36:33 \(1734410193\)
[18330.349534] Lustre: DEBUG MARKER: == sanity-lnet test 13: Delete all secondary nids from peer (tcp and o2ib) ========================================================== 04:36:33 (1734410193)
[18330.366113] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18330.371972] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18330.383379] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib
[18330.390551] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 13.13.13.13@o2ib --nid [14-15].[1-2/1].[1].[100-254/10]@tcp,14.14.[254].14@o2ib
[18332.170884] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 14: import peer create\/delete with single nid ========================================================== 04:36:35 \(1734410195\)
[18332.522578] Lustre: DEBUG MARKER: == sanity-lnet test 14: import peer create/delete with single nid ========================================================== 04:36:35 (1734410195)
[18332.541678] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18332.547546] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18332.561325] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18332.624843] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[18334.412497] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 15: import multi peer create\/delete with single nid per peer ========================================================== 04:36:37 \(1734410197\)
[18334.769249] Lustre: DEBUG MARKER: == sanity-lnet test 15: import multi peer create/delete with single nid per peer ========================================================== 04:36:37 (1734410197)
[18334.787136] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18334.793213] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18334.825977] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18334.863054] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[18336.656860] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 16: import peer create\/delete with multiple nids ========================================================== 04:36:40 \(1734410200\)
[18337.010020] Lustre: DEBUG MARKER: == sanity-lnet test 16: import peer create/delete with multiple nids ========================================================== 04:36:40 (1734410200)
[18337.027591] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18337.033563] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18337.052067] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18337.088387] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[18338.929595] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 17: import multi peer create\/delete with multiple nids ========================================================== 04:36:42 \(1734410202\)
[18339.284161] Lustre: DEBUG MARKER: == sanity-lnet test 17: import multi peer create/delete with multiple nids ========================================================== 04:36:42 (1734410202)
[18339.300603] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18339.304005] LNet: 810131:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18339.305741] LNet: 810131:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 7 previous similar messages
[18339.310091] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18339.377208] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18339.420935] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[18341.205275] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 04:36:44 \(1734410204\)
[18341.560038] Lustre: DEBUG MARKER: == sanity-lnet test 18a: Delete a subset of nids from a single peer using import --del ========================================================== 04:36:44 (1734410204)
[18341.579054] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18341.584939] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18341.596071] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18341.604331] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[18343.394368] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 04:36:46 \(1734410206\)
[18343.753260] Lustre: DEBUG MARKER: == sanity-lnet test 18b: Delete multiple nids from multiple peers using import --del ========================================================== 04:36:46 (1734410206)
[18343.775222] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18343.781326] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18343.792869] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18343.801078] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import --del
[18345.635115] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 19: Add peer with single nid \(gni\) === 04:36:49 \(1734410209\)
[18345.987377] Lustre: DEBUG MARKER: == sanity-lnet test 19: Add peer with single nid (gni) === 04:36:49 (1734410209)
[18346.002918] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18346.009021] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18346.020258] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 19@gni
[18347.825740] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 04:36:51 \(1734410211\)
[18348.181820] Lustre: DEBUG MARKER: == sanity-lnet test 20: Add peer with gni primary and tcp, o2ib secondary ========================================================== 04:36:51 (1734410211)
[18348.200368] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18348.206309] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18348.217814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 20@gni --nid 20.20.20.20@tcp,20.20.20.20@o2ib
[18350.012979] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 21: Add peer with nidrange \(gni\) ===== 04:36:53 \(1734410213\)
[18350.365315] Lustre: DEBUG MARKER: == sanity-lnet test 21: Add peer with nidrange (gni) ===== 04:36:53 (1734410213)
[18350.383143] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18350.389391] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18350.401079] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 21@gni --nid [22-25]@gni
[18350.465297] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 21@gni --nid [21-25]@gni
[18352.261816] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 22: Delete single secondary nid from peer \(gni\) ========================================================== 04:36:55 \(1734410215\)
[18352.614811] Lustre: DEBUG MARKER: == sanity-lnet test 22: Delete single secondary nid from peer (gni) ========================================================== 04:36:55 (1734410215)
[18352.631769] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18352.637896] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18352.649549] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 22@gni --nid [24-29]@gni
[18352.655644] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 22@gni --nid 26@gni
[18354.561628] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 23: Delete all secondary nids from peer \(gni\) ========================================================== 04:36:58 \(1734410218\)
[18354.915364] Lustre: DEBUG MARKER: == sanity-lnet test 23: Delete all secondary nids from peer (gni) ========================================================== 04:36:58 (1734410218)
[18354.937086] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18354.943026] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18354.954713] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 23@gni --nid [25-29]@gni
[18354.960799] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 23@gni --nid [25-29]@gni
[18356.743388] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 24: Delete a secondary nid from peer \(tcp, o2ib and gni\) ========================================================== 04:37:00 \(1734410220\)
[18357.103024] Lustre: DEBUG MARKER: == sanity-lnet test 24: Delete a secondary nid from peer (tcp, o2ib and gni) ========================================================== 04:37:00 (1734410220)
[18357.124612] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18357.130595] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18357.142086] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 24@gni --nid [13-14/1].[13-15/2].13.13@o2ib,[15-16/3].[17-19/4].[1].[5-20/5]@tcp,[5-12/6]@gni
[18357.149122] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 24@gni --nid 5@gni,13.15.13.13@o2ib,15.17.1.15@tcp
[18358.929595] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 25: Delete all secondary nids from peer \(tcp, gni and o2ib\) ========================================================== 04:37:02 \(1734410222\)
[18359.284940] Lustre: DEBUG MARKER: == sanity-lnet test 25: Delete all secondary nids from peer (tcp, gni and o2ib) ========================================================== 04:37:02 (1734410222)
[18359.303784] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18359.309564] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18359.321265] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni
[18359.328083] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 25@gni --nid [26-27].[4-10/3].26.26@tcp,26.26.26.26@o2ib,[30-35]@gni
[18361.122491] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 04:37:04 \(1734410224\)
[18361.478535] Lustre: DEBUG MARKER: == sanity-lnet test 26: Delete peer with primary nid locked ========================================================== 04:37:04 (1734410224)
[18361.499068] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18361.505186] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18361.513858] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --lock_prim
[18361.524628] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@tcp
[18361.528114] LNetError: 814828:0:(peer.c:2184:lnet_del_peer_ni()) peer 1.1.1.1@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs
[18361.536098] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@tcp --force
[18361.541834] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show --nid 1.1.1.1@tcp
[18363.293834] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 04:37:06 \(1734410226\)
[18363.650131] Lustre: DEBUG MARKER: == sanity-lnet test 27: Import bad config should fail gracefully ========================================================== 04:37:06 (1734410226)
[18363.666392] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18363.672174] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18363.683959] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/tmp.XcIrsOql56
[18363.694939] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/tmp.XcIrsOql56
[18363.703035] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/tmp.XcIrsOql56
[18365.437817] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 28: Test peer_list =================== 04:37:08 \(1734410228\)
[18365.797275] Lustre: DEBUG MARKER: == sanity-lnet test 28: Test peer_list =================== 04:37:08 (1734410228)
[18365.814053] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18365.820028] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18365.828589] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --ni 7.7.7.7@tcp
[18365.834312] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.2@tcp --nid 7.7.7.8@tcp
[18365.841840] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer list
[18367.578054] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 04:37:11 \(1734410231\)
[18367.936312] Lustre: DEBUG MARKER: == sanity-lnet test 99a: Check various invalid inputs to lnetctl peer ========================================================== 04:37:11 (1734410231)
[18367.950263] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18367.956324] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18367.964974] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid foobar
[18367.970694] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid foobar
[18367.976271] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim_nid 1.1.1.1@o2ib
[18367.981952] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --nid 1.1.1.1@tcp
[18367.987610] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --nid 1.1.1.1@tcp
[18367.993293] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add
[18367.998505] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar
[18368.004852] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.[2-255]@tcp
[18368.011123] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1@foo
[18368.016777] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1@tcp
[18368.022371] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1@o2ib
[18368.027902] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@gni
[18368.033931] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [2-1]@gni
[18368.040097] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid [a-f/x]@gni
[18368.046164] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 256.256.256.256@tcp
[18368.052072] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.1.1.1.[2-5/f]@tcp
[18368.058427] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid 1.]2[.3.4@o2ib1.[2-4,[5-6],7-8].1.1@tcp
[18368.064930] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 1.1.1.1@tcp --nid foobar
[18368.070841] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp --gateway 1@gni
[18368.074453] LNetError: 816248:0:(router.c:630:lnet_add_route()) Cannot add route with gateway 1@gni. There is no local interface configured on LNet gni
[18369.801290] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 04:37:13 \(1734410233\)
[18370.155564] Lustre: DEBUG MARKER: == sanity-lnet test 99b: Invalid value for Multi-Rail in yaml import ========================================================== 04:37:13 (1734410233)
[18370.174032] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18370.180265] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18370.191608] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18371.985414] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 100: Add route with single gw ======== 04:37:15 \(1734410235\)
[18372.338743] Lustre: DEBUG MARKER: == sanity-lnet test 100: Add route with single gw ======== 04:37:15 (1734410235)
[18383.122665] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18383.483743] LNet: 817550:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18383.485473] LNet: 817550:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 14 previous similar messages
[18383.580354] Key type .llcrypt unregistered
[18383.581244] Key type ._llcrypt unregistered
[18385.663941] Key type ._llcrypt registered
[18385.664909] Key type .llcrypt registered
[18386.045995] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18386.553213] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18386.936104] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18388.294035] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18394.894490] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18394.898460] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18394.899649] LNet: Accept all, port 7988
[18396.143524] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18396.147439] LNet: 818465:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18396.150801] LNetError: 818011:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18396.153519] LNetError: 818011:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from up to down
[18396.155785] LNetError: 818466:0:(router.c:333:lnet_is_route_alive()) peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18396.200917] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18397.205757] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18397.569060] LNet: 818591:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18397.588148] LNet: Removed LNI 10.240.22.180@tcp
[18397.696092] Key type .llcrypt unregistered
[18397.696995] Key type ._llcrypt unregistered
[18399.933056] Key type ._llcrypt registered
[18399.934007] Key type .llcrypt registered
[18400.318417] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 101: Add route with multiple gw ====== 04:37:43 \(1734410263\)
[18400.671417] Lustre: DEBUG MARKER: == sanity-lnet test 101: Add route with multiple gw ====== 04:37:43 (1734410263)
[18400.733670] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18401.504541] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18401.514104] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18401.518039] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18401.519199] LNet: Accept all, port 7988
[18401.641325] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.[181-185]@tcp
[18401.645346] LNet: 819234:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18401.648020] LNetError: 819113:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.181@tcp has gone from up to down
[18401.651029] LNetError: 819109:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.182@tcp is being used as a gateway but routing feature is not turned on
[18401.657820] LNetError: 819236:0:(router.c:333:lnet_is_route_alive()) peer 10.240.22.183@tcp is being used as a gateway but routing feature is not turned on
[18403.460964] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 102: Delete route with single gw ===== 04:37:47 \(1734410267\)
[18403.819427] Lustre: DEBUG MARKER: == sanity-lnet test 102: Delete route with single gw ===== 04:37:47 (1734410267)
[18403.832380] LNetError: 819700:0:(router.c:333:lnet_is_route_alive()) peer 10.240.22.183@tcp is being used as a gateway but routing feature is not turned on
[18403.834860] LNetError: 819700:0:(router.c:333:lnet_is_route_alive()) Skipped 3 previous similar messages
[18412.879322] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18413.247795] LNet: 820104:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18413.264116] LNet: Removed LNI 10.240.22.180@tcp
[18413.351984] Key type .llcrypt unregistered
[18413.354664] Key type ._llcrypt unregistered
[18415.501482] Key type ._llcrypt registered
[18415.502416] Key type .llcrypt registered
[18415.879428] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18416.440476] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18416.754253] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18418.070155] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18424.475416] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18424.479379] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18424.480862] LNet: Accept all, port 7988
[18425.529242] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18425.533131] LNet: 820815:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18425.536411] LNetError: 820565:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18425.537832] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp
[18425.539105] LNetError: 820565:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from up to down
[18425.587477] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18426.584345] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18426.942624] LNet: 820941:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18426.959920] LNet: Removed LNI 10.240.22.180@tcp
[18427.055887] Key type .llcrypt unregistered
[18427.056870] Key type ._llcrypt unregistered
[18429.289662] Key type ._llcrypt registered
[18429.290596] Key type .llcrypt registered
[18429.673587] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 103: Delete route with multiple gw === 04:38:13 \(1734410293\)
[18430.029549] Lustre: DEBUG MARKER: == sanity-lnet test 103: Delete route with multiple gw === 04:38:13 (1734410293)
[18430.086371] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18430.855342] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18430.864849] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18430.868948] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18430.870120] LNet: Accept all, port 7988
[18430.960977] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp103 --gateway 10.240.22.[181-186/2]@tcp
[18430.965176] LNet: 821560:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18430.967729] LNetError: 821455:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp103 through 10.240.22.181@tcp has gone from up to down
[18430.970632] LNetError: 821451:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.183@tcp is being used as a gateway but routing feature is not turned on
[18430.976270] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route del --net tcp103 --gateway 10.240.22.[181-186/2]@tcp
[18432.799710] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 104: Set\/check response_tracking param ========================================================== 04:38:16 \(1734410296\)
[18433.155589] Lustre: DEBUG MARKER: == sanity-lnet test 104: Set/check response_tracking param ========================================================== 04:38:16 (1734410296)
[18433.173289] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.176835] LNet: 822026:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18433.184054] LNet: Removed LNI 10.240.22.180@tcp
[18433.187613] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.201745] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking -1
[18433.212668] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.218719] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.229278] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18433.239987] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.245863] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.254488] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 0
[18433.267991] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.273802] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.283878] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18433.296913] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.302701] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.311267] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 1
[18433.324758] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.330607] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.340865] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18433.353941] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.359814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.368450] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 2
[18433.381638] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.387449] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.397684] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18433.410734] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.416625] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.425290] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 3
[18433.438630] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.444573] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.454637] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18433.467861] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.473756] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.482414] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set response_tracking 4
[18433.492565] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18433.498449] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18433.508560] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[18435.277768] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 04:38:18 \(1734410298\)
[18435.634473] Lustre: DEBUG MARKER: == sanity-lnet test 105: Adding duplicate GW peer should fail ========================================================== 04:38:18 (1734410298)
[18444.777428] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18445.143486] LNet: 823024:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18445.145205] LNet: 823024:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 11 previous similar messages
[18445.235830] Key type .llcrypt unregistered
[18445.236700] Key type ._llcrypt unregistered
[18447.371006] Key type ._llcrypt registered
[18447.372005] Key type .llcrypt registered
[18447.751199] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18448.343318] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18448.669584] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18449.926700] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18456.386818] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18456.390884] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18456.392105] LNet: Accept all, port 7988
[18457.439477] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18457.443525] LNet: 823733:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18457.446788] LNetError: 823485:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18457.448707] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim 10.240.22.189@tcp
[18457.449547] LNetError: 823485:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from up to down
[18457.455210] LNetError: 823735:0:(peer.c:374:lnet_peer_ni_del_locked()) Peer NI 10.240.22.189@tcp is a gateway. Can not delete it
[18457.457372] LNetError: 823735:0:(peer.c:2176:lnet_del_peer_ni()) 10.240.22.189@tcp is a router. Can not be deleted
[18457.468836] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18457.480619] LNetError: 823748:0:(router.c:333:lnet_is_route_alive()) peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18458.485079] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18458.852197] LNet: 823841:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18458.867751] LNet: Removed LNI 10.240.22.180@tcp
[18458.967739] Key type .llcrypt unregistered
[18458.968631] Key type ._llcrypt unregistered
[18461.166957] Key type ._llcrypt registered
[18461.167910] Key type .llcrypt registered
[18461.553773] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 106: Deleting GW peer should fail ==== 04:38:45 \(1734410325\)
[18461.910568] Lustre: DEBUG MARKER: == sanity-lnet test 106: Deleting GW peer should fail ==== 04:38:45 (1734410325)
[18470.960287] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18471.367615] Key type .llcrypt unregistered
[18471.368525] Key type ._llcrypt unregistered
[18473.445763] Key type ._llcrypt registered
[18473.446676] Key type .llcrypt registered
[18473.820880] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18474.255992] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18474.561816] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18476.120120] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18482.792531] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18482.796671] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18482.797843] LNet: Accept all, port 7988
[18482.841219] Autotest: Test running for 305 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[18483.846988] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18483.850753] LNet: 825508:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18483.854002] LNetError: 825171:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18483.855798] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer del --prim 10.240.22.189@tcp
[18483.856795] LNetError: 825171:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from up to down
[18483.862423] LNetError: 825510:0:(peer.c:2176:lnet_del_peer_ni()) 10.240.22.189@tcp is a router. Can not be deleted
[18483.873535] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18483.884818] LNetError: 825523:0:(router.c:333:lnet_is_route_alive()) peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18484.874605] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18485.238765] LNet: 825616:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18485.263583] LNet: Removed LNI 10.240.22.180@tcp
[18485.363537] Key type .llcrypt unregistered
[18485.364418] Key type ._llcrypt unregistered
[18487.577190] Key type ._llcrypt registered
[18487.578110] Key type .llcrypt registered
[18487.956898] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 107: Deleting extra interface doesn\'t crash node ========================================================== 04:39:11 \(1734410351\)
[18488.314650] Lustre: DEBUG MARKER: == sanity-lnet test 107: Deleting extra interface doesn't crash node ========================================================== 04:39:11 (1734410351)
[18488.332506] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18488.332506] lctl dl | grep ' ST ' || true
[18488.395521] Key type .llcrypt unregistered
[18488.396398] Key type ._llcrypt unregistered
[18488.633017] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18488.634453] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18488.741303] Key type ._llcrypt registered
[18488.742166] Key type .llcrypt registered
[18488.834541] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18489.614648] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18489.623825] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18489.627723] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18489.628913] LNet: Accept all, port 7988
[18489.715753] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[18489.719239] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18489.972911] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp --if test1pg
[18489.976665] LNet: Removed LNI 10.1.2.3@tcp
[18490.015050] LNet: 826758:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18490.033137] LNet: Removed LNI 10.240.22.180@tcp
[18490.046925] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18490.046925] lctl dl | grep ' ST ' || true
[18490.239478] Key type .llcrypt unregistered
[18490.240353] Key type ._llcrypt unregistered
[18491.879489] Key type ._llcrypt registered
[18491.883059] Key type .llcrypt registered
[18492.263280] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 108: Check Multi-Rail setup ========== 04:39:15 \(1734410355\)
[18492.617117] Lustre: DEBUG MARKER: == sanity-lnet test 108: Check Multi-Rail setup ========== 04:39:15 (1734410355)
[18492.637308] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18492.637308] lctl dl | grep ' ST ' || true
[18492.695577] Key type .llcrypt unregistered
[18492.696465] Key type ._llcrypt unregistered
[18492.954690] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18492.956158] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18493.039326] Key type ._llcrypt registered
[18493.042896] Key type .llcrypt registered
[18493.100526] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18493.885682] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18493.894679] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18493.898501] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18493.899839] LNet: Accept all, port 7988
[18493.992468] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[18493.996353] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18494.349776] LNet: 827998:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18494.351692] LNet: Removed LNI 10.240.22.180@tcp
[18494.367542] LNet: Removed LNI 10.1.2.3@tcp
[18494.375382] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18494.375382] lctl dl | grep ' ST ' || true
[18494.555556] Key type .llcrypt unregistered
[18494.556434] Key type ._llcrypt unregistered
[18496.209237] Key type ._llcrypt registered
[18496.210198] Key type .llcrypt registered
[18496.589030] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 109: Add NI using a network interface alias \(LU-16859\) ========================================================== 04:39:20 \(1734410360\)
[18496.942254] Lustre: DEBUG MARKER: == sanity-lnet test 109: Add NI using a network interface alias (LU-16859) ========================================================== 04:39:20 (1734410360)
[18496.965297] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18496.965297] lctl dl | grep ' ST ' || true
[18497.027503] Key type .llcrypt unregistered
[18497.028394] Key type ._llcrypt unregistered
[18497.259084] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18497.260544] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18497.378639] Key type ._llcrypt registered
[18497.382158] Key type .llcrypt registered
[18497.466260] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18498.233882] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18498.243030] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg:0
[18498.246945] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180]
[18498.248276] LNet: Accept all, port 7988
[18498.494264] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[18498.497868] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18498.763503] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp --if test1pg
[18498.767031] LNet: Removed LNI 10.1.2.3@tcp
[18498.770106] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp --if test1pg:0
[18498.795513] LNet: Removed LNI 10.1.2.31@tcp
[18498.798459] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[18498.802287] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18498.898268] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg:0
[18498.901742] LNet: Added LNI 10.1.2.31@tcp [8/256/0/180]
[18499.035109] LNet: 829653:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18499.037070] LNet: Removed LNI 10.1.2.3@tcp
[18499.047618] LNet: Removed LNI 10.1.2.31@tcp
[18499.054943] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18499.054943] lctl dl | grep ' ST ' || true
[18499.227529] Key type .llcrypt unregistered
[18499.228436] Key type ._llcrypt unregistered
[18500.887335] Key type ._llcrypt registered
[18500.888308] Key type .llcrypt registered
[18501.274352] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 04:39:24 \(1734410364\)
[18501.630622] Lustre: DEBUG MARKER: == sanity-lnet test 110: Configure NI using lnetctl net add --nid ========================================================== 04:39:24 (1734410364)
[18501.688336] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18502.457962] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18502.469808] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18502.473845] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18502.475016] LNet: Accept all, port 7988
[18502.578763] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18502.582102] LNet: 830407:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18502.599476] LNet: Removed LNI 10.240.22.180@tcp
[18502.602784] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18502.611269] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --nid 10.240.22.180@tcp
[18502.615055] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18502.616417] LNet: Accept all, port 7988
[18502.714965] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --nid 10.240.22.180@tcp
[18502.723528] LNet: Removed LNI 10.240.22.180@tcp
[18502.740356] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18502.740356] lctl dl | grep ' ST ' || true
[18502.867454] Key type .llcrypt unregistered
[18502.868322] Key type ._llcrypt unregistered
[18504.520961] Key type ._llcrypt registered
[18504.521908] Key type .llcrypt registered
[18504.902992] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 111: Test many routes ================ 04:39:28 \(1734410368\)
[18505.258752] Lustre: DEBUG MARKER: == sanity-lnet test 111: Test many routes ================ 04:39:28 (1734410368)
[18514.320251] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18514.723361] Key type .llcrypt unregistered
[18514.724257] Key type ._llcrypt unregistered
[18516.856919] Key type ._llcrypt registered
[18516.860510] Key type .llcrypt registered
[18517.236411] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18517.711037] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18518.039452] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18519.488091] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18525.898535] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18525.902420] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18525.903921] LNet: Accept all, port 7988
[18526.947306] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp12 -g 10.240.22.189@tcp
[18526.950950] LNet: 832205:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18526.954217] LNetError: 831944:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18526.955864] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp13 -g 10.240.22.189@tcp
[18526.956944] LNetError: 831944:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp12 through 10.240.22.189@tcp has gone from up to down
[18526.964735] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp14 -g 10.240.22.189@tcp
[18526.970538] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp15 -g 10.240.22.189@tcp
[18526.976342] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp16 -g 10.240.22.189@tcp
[18526.982188] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp17 -g 10.240.22.189@tcp
[18526.988273] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp18 -g 10.240.22.189@tcp
[18526.994272] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp19 -g 10.240.22.189@tcp
[18527.000268] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp110 -g 10.240.22.189@tcp
[18527.006321] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp111 -g 10.240.22.189@tcp
[18527.012356] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp112 -g 10.240.22.189@tcp
[18527.018411] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp113 -g 10.240.22.189@tcp
[18527.024355] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp114 -g 10.240.22.189@tcp
[18527.030424] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp115 -g 10.240.22.189@tcp
[18527.036463] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp116 -g 10.240.22.189@tcp
[18527.042478] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp117 -g 10.240.22.189@tcp
[18527.048422] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp118 -g 10.240.22.189@tcp
[18527.054459] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp119 -g 10.240.22.189@tcp
[18527.060497] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp120 -g 10.240.22.189@tcp
[18527.066537] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp121 -g 10.240.22.189@tcp
[18527.072467] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp122 -g 10.240.22.189@tcp
[18527.078488] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp123 -g 10.240.22.189@tcp
[18527.084511] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp124 -g 10.240.22.189@tcp
[18527.090524] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp125 -g 10.240.22.189@tcp
[18527.096442] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp126 -g 10.240.22.189@tcp
[18527.102470] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp127 -g 10.240.22.189@tcp
[18527.108522] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp128 -g 10.240.22.189@tcp
[18527.114569] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp129 -g 10.240.22.189@tcp
[18527.120454] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp130 -g 10.240.22.189@tcp
[18527.126464] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp131 -g 10.240.22.189@tcp
[18527.132494] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp132 -g 10.240.22.189@tcp
[18527.138493] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp133 -g 10.240.22.189@tcp
[18527.144509] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp134 -g 10.240.22.189@tcp
[18527.150584] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp135 -g 10.240.22.189@tcp
[18527.156527] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp136 -g 10.240.22.189@tcp
[18527.162555] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp137 -g 10.240.22.189@tcp
[18527.168596] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp138 -g 10.240.22.189@tcp
[18527.174588] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp139 -g 10.240.22.189@tcp
[18527.180485] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp140 -g 10.240.22.189@tcp
[18527.186534] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp141 -g 10.240.22.189@tcp
[18527.192551] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp142 -g 10.240.22.189@tcp
[18527.198563] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp143 -g 10.240.22.189@tcp
[18527.204490] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp144 -g 10.240.22.189@tcp
[18527.210536] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp145 -g 10.240.22.189@tcp
[18527.216582] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp146 -g 10.240.22.189@tcp
[18527.222588] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp147 -g 10.240.22.189@tcp
[18527.228510] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp148 -g 10.240.22.189@tcp
[18527.234544] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp149 -g 10.240.22.189@tcp
[18527.240560] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp150 -g 10.240.22.189@tcp
[18527.246557] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp151 -g 10.240.22.189@tcp
[18527.252461] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp152 -g 10.240.22.189@tcp
[18527.258501] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp153 -g 10.240.22.189@tcp
[18527.264503] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp154 -g 10.240.22.189@tcp
[18527.270479] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp155 -g 10.240.22.189@tcp
[18527.276387] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp156 -g 10.240.22.189@tcp
[18527.282376] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp157 -g 10.240.22.189@tcp
[18527.288378] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp158 -g 10.240.22.189@tcp
[18527.294360] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp159 -g 10.240.22.189@tcp
[18527.300291] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp160 -g 10.240.22.189@tcp
[18527.306325] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp161 -g 10.240.22.189@tcp
[18527.312341] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp162 -g 10.240.22.189@tcp
[18527.318358] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp163 -g 10.240.22.189@tcp
[18527.324318] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp164 -g 10.240.22.189@tcp
[18527.330384] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp165 -g 10.240.22.189@tcp
[18527.336420] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp166 -g 10.240.22.189@tcp
[18527.342427] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp167 -g 10.240.22.189@tcp
[18527.348337] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp168 -g 10.240.22.189@tcp
[18527.354385] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp169 -g 10.240.22.189@tcp
[18527.360370] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp170 -g 10.240.22.189@tcp
[18527.366382] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp171 -g 10.240.22.189@tcp
[18527.372305] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp172 -g 10.240.22.189@tcp
[18527.378347] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp173 -g 10.240.22.189@tcp
[18527.384362] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp174 -g 10.240.22.189@tcp
[18527.390363] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp175 -g 10.240.22.189@tcp
[18527.396298] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp176 -g 10.240.22.189@tcp
[18527.402313] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp177 -g 10.240.22.189@tcp
[18527.408318] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp178 -g 10.240.22.189@tcp
[18527.414322] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp179 -g 10.240.22.189@tcp
[18527.420288] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp180 -g 10.240.22.189@tcp
[18527.426313] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp181 -g 10.240.22.189@tcp
[18527.432314] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp182 -g 10.240.22.189@tcp
[18527.438313] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp183 -g 10.240.22.189@tcp
[18527.444234] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp184 -g 10.240.22.189@tcp
[18527.448010] LNet: 832369:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18527.450346] LNet: 832369:0:(router.c:737:lnet_add_route()) Skipped 81 previous similar messages
[18527.454263] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp185 -g 10.240.22.189@tcp
[18527.458236] LNetError: 831944:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18527.460942] LNetError: 831944:0:(router.c:399:lnet_router_discovery_ping_reply()) Skipped 83 previous similar messages
[18527.461405] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp186 -g 10.240.22.189@tcp
[18527.462860] LNetError: 831944:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp185 through 10.240.22.189@tcp has gone from up to down
[18527.466708] LNetError: 831944:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) Skipped 82 previous similar messages
[18527.470660] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp187 -g 10.240.22.189@tcp
[18527.476559] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp188 -g 10.240.22.189@tcp
[18527.482459] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp189 -g 10.240.22.189@tcp
[18527.488345] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp190 -g 10.240.22.189@tcp
[18527.494400] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp191 -g 10.240.22.189@tcp
[18527.500407] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp192 -g 10.240.22.189@tcp
[18527.506408] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp193 -g 10.240.22.189@tcp
[18527.512420] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp194 -g 10.240.22.189@tcp
[18527.518419] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp195 -g 10.240.22.189@tcp
[18527.524411] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp196 -g 10.240.22.189@tcp
[18527.530448] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp197 -g 10.240.22.189@tcp
[18527.536432] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp198 -g 10.240.22.189@tcp
[18527.542424] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp199 -g 10.240.22.189@tcp
[18527.548438] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1100 -g 10.240.22.189@tcp
[18527.554463] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1101 -g 10.240.22.189@tcp
[18527.560487] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1102 -g 10.240.22.189@tcp
[18527.566558] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1103 -g 10.240.22.189@tcp
[18527.572582] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1104 -g 10.240.22.189@tcp
[18527.578604] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1105 -g 10.240.22.189@tcp
[18527.584628] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1106 -g 10.240.22.189@tcp
[18527.590669] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1107 -g 10.240.22.189@tcp
[18527.596543] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1108 -g 10.240.22.189@tcp
[18527.602584] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1109 -g 10.240.22.189@tcp
[18527.608606] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1110 -g 10.240.22.189@tcp
[18527.614584] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1111 -g 10.240.22.189@tcp
[18527.620539] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1112 -g 10.240.22.189@tcp
[18527.626540] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1113 -g 10.240.22.189@tcp
[18527.632549] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1114 -g 10.240.22.189@tcp
[18527.638543] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1115 -g 10.240.22.189@tcp
[18527.644449] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1116 -g 10.240.22.189@tcp
[18527.650492] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1117 -g 10.240.22.189@tcp
[18527.656514] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1118 -g 10.240.22.189@tcp
[18527.662570] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1119 -g 10.240.22.189@tcp
[18527.668474] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1120 -g 10.240.22.189@tcp
[18527.674476] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1121 -g 10.240.22.189@tcp
[18527.680534] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1122 -g 10.240.22.189@tcp
[18527.686526] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1123 -g 10.240.22.189@tcp
[18527.692487] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1124 -g 10.240.22.189@tcp
[18527.698572] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1125 -g 10.240.22.189@tcp
[18527.704614] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1126 -g 10.240.22.189@tcp
[18527.710659] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1127 -g 10.240.22.189@tcp
[18527.716573] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1128 -g 10.240.22.189@tcp
[18527.722558] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1129 -g 10.240.22.189@tcp
[18527.728568] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1130 -g 10.240.22.189@tcp
[18527.734570] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1131 -g 10.240.22.189@tcp
[18527.740456] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1132 -g 10.240.22.189@tcp
[18527.746482] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1133 -g 10.240.22.189@tcp
[18527.752543] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1134 -g 10.240.22.189@tcp
[18527.758562] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1135 -g 10.240.22.189@tcp
[18527.764476] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1136 -g 10.240.22.189@tcp
[18527.770516] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1137 -g 10.240.22.189@tcp
[18527.776539] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1138 -g 10.240.22.189@tcp
[18527.782551] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1139 -g 10.240.22.189@tcp
[18527.788464] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1140 -g 10.240.22.189@tcp
[18527.794521] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1141 -g 10.240.22.189@tcp
[18527.800574] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1142 -g 10.240.22.189@tcp
[18527.806576] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1143 -g 10.240.22.189@tcp
[18527.812485] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1144 -g 10.240.22.189@tcp
[18527.818481] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1145 -g 10.240.22.189@tcp
[18527.824489] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1146 -g 10.240.22.189@tcp
[18527.830507] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1147 -g 10.240.22.189@tcp
[18527.836489] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1148 -g 10.240.22.189@tcp
[18527.842501] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1149 -g 10.240.22.189@tcp
[18527.848503] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1150 -g 10.240.22.189@tcp
[18527.854533] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1151 -g 10.240.22.189@tcp
[18527.860481] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1152 -g 10.240.22.189@tcp
[18527.866511] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1153 -g 10.240.22.189@tcp
[18527.872563] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1154 -g 10.240.22.189@tcp
[18527.878556] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1155 -g 10.240.22.189@tcp
[18527.884490] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1156 -g 10.240.22.189@tcp
[18527.890501] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1157 -g 10.240.22.189@tcp
[18527.896525] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1158 -g 10.240.22.189@tcp
[18527.902550] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1159 -g 10.240.22.189@tcp
[18527.908475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1160 -g 10.240.22.189@tcp
[18527.914481] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1161 -g 10.240.22.189@tcp
[18527.920465] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1162 -g 10.240.22.189@tcp
[18527.926473] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1163 -g 10.240.22.189@tcp
[18527.932419] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1164 -g 10.240.22.189@tcp
[18527.938493] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1165 -g 10.240.22.189@tcp
[18527.944536] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1166 -g 10.240.22.189@tcp
[18527.950527] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1167 -g 10.240.22.189@tcp
[18527.956466] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1168 -g 10.240.22.189@tcp
[18527.962508] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1169 -g 10.240.22.189@tcp
[18527.968627] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1170 -g 10.240.22.189@tcp
[18527.974498] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1171 -g 10.240.22.189@tcp
[18527.980317] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1172 -g 10.240.22.189@tcp
[18527.986220] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1173 -g 10.240.22.189@tcp
[18527.992120] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1174 -g 10.240.22.189@tcp
[18527.998044] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1175 -g 10.240.22.189@tcp
[18528.003855] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1176 -g 10.240.22.189@tcp
[18528.009744] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1177 -g 10.240.22.189@tcp
[18528.015676] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1178 -g 10.240.22.189@tcp
[18528.021486] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1179 -g 10.240.22.189@tcp
[18528.027294] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1180 -g 10.240.22.189@tcp
[18528.033069] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1181 -g 10.240.22.189@tcp
[18528.038902] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1182 -g 10.240.22.189@tcp
[18528.044815] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1183 -g 10.240.22.189@tcp
[18528.050631] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1184 -g 10.240.22.189@tcp
[18528.056479] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1185 -g 10.240.22.189@tcp
[18528.062339] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1186 -g 10.240.22.189@tcp
[18528.068301] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1187 -g 10.240.22.189@tcp
[18528.074172] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1188 -g 10.240.22.189@tcp
[18528.080012] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1189 -g 10.240.22.189@tcp
[18528.085876] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1190 -g 10.240.22.189@tcp
[18528.091737] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1191 -g 10.240.22.189@tcp
[18528.097571] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1192 -g 10.240.22.189@tcp
[18528.103403] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1193 -g 10.240.22.189@tcp
[18528.109176] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1194 -g 10.240.22.189@tcp
[18528.115135] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1195 -g 10.240.22.189@tcp
[18528.120979] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1196 -g 10.240.22.189@tcp
[18528.126909] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1197 -g 10.240.22.189@tcp
[18528.132872] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1198 -g 10.240.22.189@tcp
[18528.138718] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1199 -g 10.240.22.189@tcp
[18528.144679] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1200 -g 10.240.22.189@tcp
[18528.150592] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1201 -g 10.240.22.189@tcp
[18528.156632] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1202 -g 10.240.22.189@tcp
[18528.162626] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1203 -g 10.240.22.189@tcp
[18528.168655] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1204 -g 10.240.22.189@tcp
[18528.174566] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1205 -g 10.240.22.189@tcp
[18528.180603] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1206 -g 10.240.22.189@tcp
[18528.186513] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1207 -g 10.240.22.189@tcp
[18528.192450] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1208 -g 10.240.22.189@tcp
[18528.198387] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1209 -g 10.240.22.189@tcp
[18528.204457] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1210 -g 10.240.22.189@tcp
[18528.210444] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1211 -g 10.240.22.189@tcp
[18528.216394] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1212 -g 10.240.22.189@tcp
[18528.222332] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1213 -g 10.240.22.189@tcp
[18528.228447] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1214 -g 10.240.22.189@tcp
[18528.234405] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1215 -g 10.240.22.189@tcp
[18528.240406] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1216 -g 10.240.22.189@tcp
[18528.246419] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1217 -g 10.240.22.189@tcp
[18528.252378] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1218 -g 10.240.22.189@tcp
[18528.258343] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1219 -g 10.240.22.189@tcp
[18528.264395] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1220 -g 10.240.22.189@tcp
[18528.270351] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1221 -g 10.240.22.189@tcp
[18528.276296] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1222 -g 10.240.22.189@tcp
[18528.282308] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1223 -g 10.240.22.189@tcp
[18528.288285] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1224 -g 10.240.22.189@tcp
[18528.294220] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1225 -g 10.240.22.189@tcp
[18528.300282] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1226 -g 10.240.22.189@tcp
[18528.306201] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1227 -g 10.240.22.189@tcp
[18528.312144] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1228 -g 10.240.22.189@tcp
[18528.318123] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1229 -g 10.240.22.189@tcp
[18528.324127] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1230 -g 10.240.22.189@tcp
[18528.330043] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1231 -g 10.240.22.189@tcp
[18528.336109] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1232 -g 10.240.22.189@tcp
[18528.342054] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1233 -g 10.240.22.189@tcp
[18528.348039] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1234 -g 10.240.22.189@tcp
[18528.354053] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1235 -g 10.240.22.189@tcp
[18528.360095] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1236 -g 10.240.22.189@tcp
[18528.366050] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1237 -g 10.240.22.189@tcp
[18528.372153] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1238 -g 10.240.22.189@tcp
[18528.378090] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1239 -g 10.240.22.189@tcp
[18528.384058] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1240 -g 10.240.22.189@tcp
[18528.390082] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1241 -g 10.240.22.189@tcp
[18528.396084] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1242 -g 10.240.22.189@tcp
[18528.402032] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1243 -g 10.240.22.189@tcp
[18528.408093] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1244 -g 10.240.22.189@tcp
[18528.414036] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1245 -g 10.240.22.189@tcp
[18528.420046] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1246 -g 10.240.22.189@tcp
[18528.426046] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1247 -g 10.240.22.189@tcp
[18528.432048] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1248 -g 10.240.22.189@tcp
[18528.437958] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1249 -g 10.240.22.189@tcp
[18528.444008] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1250 -g 10.240.22.189@tcp
[18528.447803] LNet: 832701:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18528.450108] LNet: 832701:0:(router.c:737:lnet_add_route()) Skipped 165 previous similar messages
[18528.453962] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1251 -g 10.240.22.189@tcp
[18528.458013] LNetError: 831944:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18528.460733] LNetError: 831944:0:(router.c:399:lnet_router_discovery_ping_reply()) Skipped 164 previous similar messages
[18528.462417] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1252 -g 10.240.22.189@tcp
[18528.466682] LNetError: 831944:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1252 through 10.240.22.189@tcp has gone from up to down
[18528.469023] LNetError: 831944:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) Skipped 166 previous similar messages
[18528.469372] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1253 -g 10.240.22.189@tcp
[18528.476974] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1254 -g 10.240.22.189@tcp
[18528.482898] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1255 -g 10.240.22.189@tcp
[18528.488857] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1256 -g 10.240.22.189@tcp
[18528.494953] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1257 -g 10.240.22.189@tcp
[18528.500967] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1258 -g 10.240.22.189@tcp
[18528.506979] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1259 -g 10.240.22.189@tcp
[18528.513068] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1260 -g 10.240.22.189@tcp
[18528.519109] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1261 -g 10.240.22.189@tcp
[18528.525238] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1262 -g 10.240.22.189@tcp
[18528.531173] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1263 -g 10.240.22.189@tcp
[18528.537063] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1264 -g 10.240.22.189@tcp
[18528.543071] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1265 -g 10.240.22.189@tcp
[18528.548965] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1266 -g 10.240.22.189@tcp
[18528.554899] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1267 -g 10.240.22.189@tcp
[18528.560980] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1268 -g 10.240.22.189@tcp
[18528.566893] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1269 -g 10.240.22.189@tcp
[18528.572865] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1270 -g 10.240.22.189@tcp
[18528.578846] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1271 -g 10.240.22.189@tcp
[18528.584899] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1272 -g 10.240.22.189@tcp
[18528.590953] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1273 -g 10.240.22.189@tcp
[18528.596960] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1274 -g 10.240.22.189@tcp
[18528.602937] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1275 -g 10.240.22.189@tcp
[18528.609062] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1276 -g 10.240.22.189@tcp
[18528.615028] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1277 -g 10.240.22.189@tcp
[18528.620980] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1278 -g 10.240.22.189@tcp
[18528.626923] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1279 -g 10.240.22.189@tcp
[18528.632979] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1280 -g 10.240.22.189@tcp
[18528.638861] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1281 -g 10.240.22.189@tcp
[18528.644857] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1282 -g 10.240.22.189@tcp
[18528.650914] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1283 -g 10.240.22.189@tcp
[18528.656914] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1284 -g 10.240.22.189@tcp
[18528.662866] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1285 -g 10.240.22.189@tcp
[18528.668961] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1286 -g 10.240.22.189@tcp
[18528.674865] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1287 -g 10.240.22.189@tcp
[18528.680851] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1288 -g 10.240.22.189@tcp
[18528.686823] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1289 -g 10.240.22.189@tcp
[18528.692882] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1290 -g 10.240.22.189@tcp
[18528.698912] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1291 -g 10.240.22.189@tcp
[18528.704965] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1292 -g 10.240.22.189@tcp
[18528.710866] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1293 -g 10.240.22.189@tcp
[18528.716869] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1294 -g 10.240.22.189@tcp
[18528.722714] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1295 -g 10.240.22.189@tcp
[18528.728703] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1296 -g 10.240.22.189@tcp
[18528.734660] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1297 -g 10.240.22.189@tcp
[18528.740723] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1298 -g 10.240.22.189@tcp
[18528.746763] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1299 -g 10.240.22.189@tcp
[18528.752813] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1300 -g 10.240.22.189@tcp
[18528.758780] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1301 -g 10.240.22.189@tcp
[18528.764882] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1302 -g 10.240.22.189@tcp
[18528.770931] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1303 -g 10.240.22.189@tcp
[18528.776903] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1304 -g 10.240.22.189@tcp
[18528.782859] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1305 -g 10.240.22.189@tcp
[18528.788986] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1306 -g 10.240.22.189@tcp
[18528.795000] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1307 -g 10.240.22.189@tcp
[18528.800999] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1308 -g 10.240.22.189@tcp
[18528.806936] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1309 -g 10.240.22.189@tcp
[18528.812851] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1310 -g 10.240.22.189@tcp
[18528.818982] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1311 -g 10.240.22.189@tcp
[18528.824961] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1312 -g 10.240.22.189@tcp
[18528.830933] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1313 -g 10.240.22.189@tcp
[18528.837005] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1314 -g 10.240.22.189@tcp
[18528.842965] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1315 -g 10.240.22.189@tcp
[18528.848977] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1316 -g 10.240.22.189@tcp
[18528.854870] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1317 -g 10.240.22.189@tcp
[18528.860879] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1318 -g 10.240.22.189@tcp
[18528.866814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1319 -g 10.240.22.189@tcp
[18528.872842] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1320 -g 10.240.22.189@tcp
[18528.878803] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1321 -g 10.240.22.189@tcp
[18528.884927] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1322 -g 10.240.22.189@tcp
[18528.890887] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1323 -g 10.240.22.189@tcp
[18528.896871] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1324 -g 10.240.22.189@tcp
[18528.902900] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1325 -g 10.240.22.189@tcp
[18528.908898] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1326 -g 10.240.22.189@tcp
[18528.914876] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1327 -g 10.240.22.189@tcp
[18528.920970] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1328 -g 10.240.22.189@tcp
[18528.926925] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1329 -g 10.240.22.189@tcp
[18528.932907] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1330 -g 10.240.22.189@tcp
[18528.938906] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1331 -g 10.240.22.189@tcp
[18528.944971] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1332 -g 10.240.22.189@tcp
[18528.950903] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1333 -g 10.240.22.189@tcp
[18528.956963] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1334 -g 10.240.22.189@tcp
[18528.962994] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1335 -g 10.240.22.189@tcp
[18528.968990] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1336 -g 10.240.22.189@tcp
[18528.974955] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1337 -g 10.240.22.189@tcp
[18528.981033] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1338 -g 10.240.22.189@tcp
[18528.987000] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1339 -g 10.240.22.189@tcp
[18528.992853] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1340 -g 10.240.22.189@tcp
[18528.998749] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1341 -g 10.240.22.189@tcp
[18529.004671] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1342 -g 10.240.22.189@tcp
[18529.010616] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1343 -g 10.240.22.189@tcp
[18529.016507] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1344 -g 10.240.22.189@tcp
[18529.022477] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1345 -g 10.240.22.189@tcp
[18529.028394] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1346 -g 10.240.22.189@tcp
[18529.034324] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1347 -g 10.240.22.189@tcp
[18529.040255] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1348 -g 10.240.22.189@tcp
[18529.046152] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1349 -g 10.240.22.189@tcp
[18529.052009] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1350 -g 10.240.22.189@tcp
[18529.057939] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1351 -g 10.240.22.189@tcp
[18529.063848] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1352 -g 10.240.22.189@tcp
[18529.069863] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1353 -g 10.240.22.189@tcp
[18529.075929] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1354 -g 10.240.22.189@tcp
[18529.081971] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1355 -g 10.240.22.189@tcp
[18529.087996] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1356 -g 10.240.22.189@tcp
[18529.093988] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1357 -g 10.240.22.189@tcp
[18529.100023] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1358 -g 10.240.22.189@tcp
[18529.106072] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1359 -g 10.240.22.189@tcp
[18529.112109] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1360 -g 10.240.22.189@tcp
[18529.118134] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1361 -g 10.240.22.189@tcp
[18529.124187] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1362 -g 10.240.22.189@tcp
[18529.130236] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1363 -g 10.240.22.189@tcp
[18529.136244] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1364 -g 10.240.22.189@tcp
[18529.142252] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1365 -g 10.240.22.189@tcp
[18529.148282] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1366 -g 10.240.22.189@tcp
[18529.154396] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1367 -g 10.240.22.189@tcp
[18529.160419] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1368 -g 10.240.22.189@tcp
[18529.166442] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1369 -g 10.240.22.189@tcp
[18529.172461] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1370 -g 10.240.22.189@tcp
[18529.178449] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1371 -g 10.240.22.189@tcp
[18529.184446] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1372 -g 10.240.22.189@tcp
[18529.190519] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1373 -g 10.240.22.189@tcp
[18529.196522] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1374 -g 10.240.22.189@tcp
[18529.202523] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1375 -g 10.240.22.189@tcp
[18529.208518] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1376 -g 10.240.22.189@tcp
[18529.214536] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1377 -g 10.240.22.189@tcp
[18529.220514] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1378 -g 10.240.22.189@tcp
[18529.226502] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1379 -g 10.240.22.189@tcp
[18529.232532] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1380 -g 10.240.22.189@tcp
[18529.238576] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1381 -g 10.240.22.189@tcp
[18529.244588] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1382 -g 10.240.22.189@tcp
[18529.250584] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1383 -g 10.240.22.189@tcp
[18529.256571] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1384 -g 10.240.22.189@tcp
[18529.262585] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1385 -g 10.240.22.189@tcp
[18529.268587] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1386 -g 10.240.22.189@tcp
[18529.274660] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1387 -g 10.240.22.189@tcp
[18529.280685] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1388 -g 10.240.22.189@tcp
[18529.286700] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1389 -g 10.240.22.189@tcp
[18529.292725] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1390 -g 10.240.22.189@tcp
[18529.298700] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1391 -g 10.240.22.189@tcp
[18529.304751] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1392 -g 10.240.22.189@tcp
[18529.310757] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1393 -g 10.240.22.189@tcp
[18529.316795] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1394 -g 10.240.22.189@tcp
[18529.322794] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1395 -g 10.240.22.189@tcp
[18529.328803] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1396 -g 10.240.22.189@tcp
[18529.334789] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1397 -g 10.240.22.189@tcp
[18529.340789] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1398 -g 10.240.22.189@tcp
[18529.346787] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1399 -g 10.240.22.189@tcp
[18529.352805] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1400 -g 10.240.22.189@tcp
[18529.358829] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1401 -g 10.240.22.189@tcp
[18529.364814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1402 -g 10.240.22.189@tcp
[18529.370824] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1403 -g 10.240.22.189@tcp
[18529.376811] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1404 -g 10.240.22.189@tcp
[18529.382782] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1405 -g 10.240.22.189@tcp
[18529.388791] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1406 -g 10.240.22.189@tcp
[18529.394817] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1407 -g 10.240.22.189@tcp
[18529.400804] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1408 -g 10.240.22.189@tcp
[18529.406779] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1409 -g 10.240.22.189@tcp
[18529.412808] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1410 -g 10.240.22.189@tcp
[18529.418811] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1411 -g 10.240.22.189@tcp
[18529.424808] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1412 -g 10.240.22.189@tcp
[18529.430806] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1413 -g 10.240.22.189@tcp
[18529.436936] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1414 -g 10.240.22.189@tcp
[18529.443000] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1415 -g 10.240.22.189@tcp
[18529.449076] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1416 -g 10.240.22.189@tcp
[18529.455086] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1417 -g 10.240.22.189@tcp
[18529.461085] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1418 -g 10.240.22.189@tcp
[18529.467160] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1419 -g 10.240.22.189@tcp
[18529.473122] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1420 -g 10.240.22.189@tcp
[18529.479105] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1421 -g 10.240.22.189@tcp
[18529.485087] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1422 -g 10.240.22.189@tcp
[18529.491061] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1423 -g 10.240.22.189@tcp
[18529.497080] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1424 -g 10.240.22.189@tcp
[18529.503147] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1425 -g 10.240.22.189@tcp
[18529.509242] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1426 -g 10.240.22.189@tcp
[18529.515103] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1427 -g 10.240.22.189@tcp
[18529.520992] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1428 -g 10.240.22.189@tcp
[18529.526847] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1429 -g 10.240.22.189@tcp
[18529.532610] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1430 -g 10.240.22.189@tcp
[18529.538418] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1431 -g 10.240.22.189@tcp
[18529.544365] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1432 -g 10.240.22.189@tcp
[18529.550298] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1433 -g 10.240.22.189@tcp
[18529.556107] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1434 -g 10.240.22.189@tcp
[18529.562041] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1435 -g 10.240.22.189@tcp
[18529.567943] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1436 -g 10.240.22.189@tcp
[18529.573818] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1437 -g 10.240.22.189@tcp
[18529.579644] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1438 -g 10.240.22.189@tcp
[18529.585563] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1439 -g 10.240.22.189@tcp
[18529.591516] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1440 -g 10.240.22.189@tcp
[18529.597401] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1441 -g 10.240.22.189@tcp
[18529.603253] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1442 -g 10.240.22.189@tcp
[18529.609115] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1443 -g 10.240.22.189@tcp
[18529.614883] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1444 -g 10.240.22.189@tcp
[18529.620708] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1445 -g 10.240.22.189@tcp
[18529.626547] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1446 -g 10.240.22.189@tcp
[18529.632360] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1447 -g 10.240.22.189@tcp
[18529.638301] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1448 -g 10.240.22.189@tcp
[18529.644240] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1449 -g 10.240.22.189@tcp
[18529.650168] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1450 -g 10.240.22.189@tcp
[18529.655991] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1451 -g 10.240.22.189@tcp
[18529.661911] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1452 -g 10.240.22.189@tcp
[18529.667822] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1453 -g 10.240.22.189@tcp
[18529.673707] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1454 -g 10.240.22.189@tcp
[18529.679553] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1455 -g 10.240.22.189@tcp
[18529.685486] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1456 -g 10.240.22.189@tcp
[18529.691465] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1457 -g 10.240.22.189@tcp
[18529.697421] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1458 -g 10.240.22.189@tcp
[18529.703425] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1459 -g 10.240.22.189@tcp
[18529.709310] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1460 -g 10.240.22.189@tcp
[18529.715093] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1461 -g 10.240.22.189@tcp
[18529.720988] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1462 -g 10.240.22.189@tcp
[18529.726934] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1463 -g 10.240.22.189@tcp
[18529.732820] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1464 -g 10.240.22.189@tcp
[18529.738759] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1465 -g 10.240.22.189@tcp
[18529.744548] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1466 -g 10.240.22.189@tcp
[18529.750457] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1467 -g 10.240.22.189@tcp
[18529.756387] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1468 -g 10.240.22.189@tcp
[18529.762341] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1469 -g 10.240.22.189@tcp
[18529.768195] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1470 -g 10.240.22.189@tcp
[18529.774080] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1471 -g 10.240.22.189@tcp
[18529.780003] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1472 -g 10.240.22.189@tcp
[18529.785911] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1473 -g 10.240.22.189@tcp
[18529.791852] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1474 -g 10.240.22.189@tcp
[18529.797786] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1475 -g 10.240.22.189@tcp
[18529.803850] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1476 -g 10.240.22.189@tcp
[18529.809781] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1477 -g 10.240.22.189@tcp
[18529.815764] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1478 -g 10.240.22.189@tcp
[18529.821759] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1479 -g 10.240.22.189@tcp
[18529.827743] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1480 -g 10.240.22.189@tcp
[18529.833684] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1481 -g 10.240.22.189@tcp
[18529.839748] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1482 -g 10.240.22.189@tcp
[18529.845669] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1483 -g 10.240.22.189@tcp
[18529.851736] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1484 -g 10.240.22.189@tcp
[18529.857893] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1485 -g 10.240.22.189@tcp
[18529.863843] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1486 -g 10.240.22.189@tcp
[18529.869763] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1487 -g 10.240.22.189@tcp
[18529.875810] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1488 -g 10.240.22.189@tcp
[18529.881731] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1489 -g 10.240.22.189@tcp
[18529.887729] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1490 -g 10.240.22.189@tcp
[18529.893717] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1491 -g 10.240.22.189@tcp
[18529.899703] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1492 -g 10.240.22.189@tcp
[18529.905626] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1493 -g 10.240.22.189@tcp
[18529.911561] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1494 -g 10.240.22.189@tcp
[18529.917405] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1495 -g 10.240.22.189@tcp
[18529.923342] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1496 -g 10.240.22.189@tcp
[18529.929131] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1497 -g 10.240.22.189@tcp
[18529.935163] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1498 -g 10.240.22.189@tcp
[18529.940965] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1499 -g 10.240.22.189@tcp
[18529.946742] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add -n tcp1500 -g 10.240.22.189@tcp
[18529.954351] LNetError: 833203:0:(router.c:333:lnet_is_route_alive()) peer 10.240.22.189@tcp is being used as a gateway but routing feature is not turned on
[18529.969832] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18530.972336] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18531.336269] LNet: 833309:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18531.351296] LNet: Removed LNI 10.240.22.180@tcp
[18531.447773] Key type .llcrypt unregistered
[18531.448674] Key type ._llcrypt unregistered
[18533.683320] Key type ._llcrypt registered
[18533.689527] Key type .llcrypt registered
[18534.068884] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 112: multiple net configurations ===== 04:39:57 \(1734410397\)
[18534.420672] Lustre: DEBUG MARKER: == sanity-lnet test 112: multiple net configurations ===== 04:39:57 (1734410397)
[18534.439793] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18534.439793] lctl dl | grep ' ST ' || true
[18534.499250] Key type .llcrypt unregistered
[18534.500119] Key type ._llcrypt unregistered
[18534.667649] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18534.669004] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18534.773624] Key type ._llcrypt registered
[18534.774549] Key type .llcrypt registered
[18534.870638] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18535.644299] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18535.653444] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18535.654632] LNet: Accept all, port 7988
[18535.748149] LNet: Added LNI 10.1.2.3@tcp2 [8/256/0/180]
[18536.019296] LNet: 834409:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18536.021072] LNet: Removed LNI 10.240.22.180@tcp1
[18536.039254] LNet: Removed LNI 10.1.2.3@tcp2
[18536.045745] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18536.045745] lctl dl | grep ' ST ' || true
[18536.175240] Key type .llcrypt unregistered
[18536.176101] Key type ._llcrypt unregistered
[18537.903811] Key type ._llcrypt registered
[18537.904739] Key type .llcrypt registered
[18538.287363] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 199: load lnet w\/o module option, configure interface twice ========================================================== 04:40:01 \(1734410401\)
[18538.646858] Lustre: DEBUG MARKER: == sanity-lnet test 199: load lnet w/o module option, configure interface twice ========================================================== 04:40:01 (1734410401)
[18538.711203] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18539.476200] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18539.485310] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18539.489206] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18539.490371] LNet: Accept all, port 7988
[18539.590879] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --ip2net tcp 10.240.22.180
[18539.594418] LNet: Removed LNI 10.240.22.180@tcp
[18541.334257] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 200: load lnet w\/o module option, configure in a non-default namespace ========================================================== 04:40:04 \(1734410404\)
[18541.693389] Lustre: DEBUG MARKER: == sanity-lnet test 200: load lnet w/o module option, configure in a non-default namespace ========================================================== 04:40:04 (1734410404)
[18541.709194] LNet: 835632:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18541.727272] LNet: Removed LNI 10.240.22.180@tcp
[18541.733661] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18541.733661] lctl dl | grep ' ST ' || true
[18541.859283] Key type .llcrypt unregistered
[18541.860159] Key type ._llcrypt unregistered
[18542.070443] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18542.071876] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18542.175874] Key type ._llcrypt registered
[18542.176754] Key type .llcrypt registered
[18542.229342] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18542.998414] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18542.999586] LNet: Accept all, port 7988
[18544.798053] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 04:40:08 \(1734410408\)
[18545.158226] Lustre: DEBUG MARKER: == sanity-lnet test 201: load lnet using networks module options in a non-default namespace ========================================================== 04:40:08 (1734410408)
[18545.174000] LNet: 836325:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18545.187251] LNet: Removed LNI 10.1.2.3@tcp
[18545.194009] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18545.194009] lctl dl | grep ' ST ' || true
[18545.307251] Key type .llcrypt unregistered
[18545.308117] Key type ._llcrypt unregistered
[18545.523031] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18545.524473] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18545.645242] Key type ._llcrypt registered
[18545.646118] Key type .llcrypt registered
[18545.720822] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18546.494795] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18546.496035] LNet: Accept all, port 7988
[18548.319493] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 04:40:11 \(1734410411\)
[18548.673473] Lustre: DEBUG MARKER: == sanity-lnet test 202: load lnet using ip2nets in a non-default namespace ========================================================== 04:40:11 (1734410411)
[18548.691090] LNet: 837018:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18548.703284] LNet: Removed LNI 10.1.2.3@tcp
[18548.709781] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18548.709781] lctl dl | grep ' ST ' || true
[18548.839170] Key type .llcrypt unregistered
[18548.840059] Key type ._llcrypt unregistered
[18549.102956] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18549.104369] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18549.179594] Key type ._llcrypt registered
[18549.180468] Key type .llcrypt registered
[18549.291920] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18550.067478] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18550.068684] LNet: Accept all, port 7988
[18551.853200] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 04:40:15 \(1734410415\)
[18552.210400] Lustre: DEBUG MARKER: == sanity-lnet test 203: add a network using an interface in the non-default namespace ========================================================== 04:40:15 (1734410415)
[18552.234062] LNet: 837711:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18552.255230] LNet: Removed LNI 10.1.2.3@tcp
[18552.261798] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18552.261798] lctl dl | grep ' ST ' || true
[18552.387159] Key type .llcrypt unregistered
[18552.388052] Key type ._llcrypt unregistered
[18552.630331] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18552.631766] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18552.734724] Key type ._llcrypt registered
[18552.739274] Key type .llcrypt registered
[18552.802567] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18553.594395] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18553.602675] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18553.603776] LNet: Accept all, port 7988
[18555.654296] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 04:40:19 \(1734410419\)
[18556.008103] Lustre: DEBUG MARKER: == sanity-lnet test 204: Check no health or resends for single-rail local failures ========================================================== 04:40:19 (1734410419)
[18556.080414] LNet: 838747:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18556.095217] LNet: Removed LNI 10.1.2.3@tcp
[18556.101513] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18556.101513] lctl dl | grep ' ST ' || true
[18556.223124] Key type .llcrypt unregistered
[18556.223998] Key type ._llcrypt unregistered
[18556.472537] Key type ._llcrypt registered
[18556.473452] Key type .llcrypt registered
[18556.513084] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18557.284297] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18557.288575] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18557.289739] LNet: Accept all, port 7988
[18560.992176] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18561.063605] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18561.066987] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18561.107609] LNet: 839049:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18561.119214] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18561.124806] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18561.181648] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18561.233950] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18561.239643] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18561.296511] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18561.349215] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18561.358137] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18561.415577] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18561.467622] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18561.473312] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18561.530663] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18561.583704] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18561.589311] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18561.646791] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18561.650582] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18561.652259] LNet: Skipped 4 previous similar messages
[18561.692198] LNet: 839389:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18561.694053] LNet: 839389:0:(api-ni.c:337:recovery_interval_set()) Skipped 4 previous similar messages
[18561.705426] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18561.711068] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18561.722309] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18561.722309] lctl dl | grep ' ST ' || true
[18561.750357] LNet: 839418:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18562.795094] LNet: Removed LNI 10.240.22.180@tcp
[18562.891092] Key type .llcrypt unregistered
[18562.891966] Key type ._llcrypt unregistered
[18566.687586] Key type ._llcrypt registered
[18566.688514] Key type .llcrypt registered
[18567.066245] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 04:40:30 \(1734410430\)
[18567.418136] Lustre: DEBUG MARKER: == sanity-lnet test 205: Check health and resends for multi-rail local failures ========================================================== 04:40:30 (1734410430)
[18567.496189] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18567.496189] lctl dl | grep ' ST ' || true
[18567.563049] Key type .llcrypt unregistered
[18567.563926] Key type ._llcrypt unregistered
[18567.796490] Key type ._llcrypt registered
[18567.797449] Key type .llcrypt registered
[18567.844121] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18568.612779] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18568.617461] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18568.618622] LNet: Accept all, port 7988
[18572.316502] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18576.453194] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18576.459122] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18576.462759] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18576.640684] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18576.644228] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18576.645950] LNet: 1 local NIs in recovery (showing 1): 10.240.22.180@tcp
[18576.688043] LNet: 840542:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18576.699560] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18576.705300] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18576.774531] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18576.828096] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18576.833854] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18576.904057] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18576.957362] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18576.963043] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18577.034562] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18577.088716] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18577.094443] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18577.165315] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18577.169231] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18577.170907] LNet: Skipped 12 previous similar messages
[18577.211973] LNet: 840838:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18577.213820] LNet: 840838:0:(api-ni.c:337:recovery_interval_set()) Skipped 3 previous similar messages
[18577.225339] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18577.230965] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18577.302240] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18577.355186] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18577.360858] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18580.345946] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18580.345946] lctl dl | grep ' ST ' || true
[18580.374829] LNet: 840978:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18580.376664] LNet: Removed LNI 10.240.22.180@tcp
[18580.387006] LNet: Removed LNI 10.240.22.180@tcp1
[18580.483064] Key type .llcrypt unregistered
[18580.483937] Key type ._llcrypt unregistered
[18584.348054] Key type ._llcrypt registered
[18584.349087] Key type .llcrypt registered
[18584.731592] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 04:40:48 \(1734410448\)
[18585.082782] Lustre: DEBUG MARKER: == sanity-lnet test 206: Check no health or resends for single-rail remote failures ========================================================== 04:40:48 (1734410448)
[18585.155625] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18585.155625] lctl dl | grep ' ST ' || true
[18585.218935] Key type .llcrypt unregistered
[18585.219815] Key type ._llcrypt unregistered
[18585.392931] Key type ._llcrypt registered
[18585.393818] Key type .llcrypt registered
[18585.434893] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18586.209811] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18586.214176] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18586.215589] LNet: Accept all, port 7988
[18589.919188] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18589.991050] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18589.994531] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18590.035314] LNet: 841891:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18590.046867] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18590.052549] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18590.109521] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18590.161533] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18590.167203] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18590.225299] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18590.276835] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18590.282500] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18590.294247] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18590.294247] lctl dl | grep ' ST ' || true
[18590.322327] LNet: 842056:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18590.334944] LNet: Removed LNI 10.240.22.180@tcp
[18590.434972] Key type .llcrypt unregistered
[18590.435848] Key type ._llcrypt unregistered
[18594.252445] Key type ._llcrypt registered
[18594.253375] Key type .llcrypt registered
[18594.633694] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 04:40:58 \(1734410458\)
[18594.990150] Lustre: DEBUG MARKER: == sanity-lnet test 207: Check health and resends for multi-rail remote errors ========================================================== 04:40:58 (1734410458)
[18595.068222] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18595.068222] lctl dl | grep ' ST ' || true
[18595.130887] Key type .llcrypt unregistered
[18595.131747] Key type ._llcrypt unregistered
[18595.335020] Key type ._llcrypt registered
[18595.335927] Key type .llcrypt registered
[18595.413990] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18596.184408] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18596.188821] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18596.189987] LNet: Accept all, port 7988
[18599.901897] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18604.061177] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18604.066935] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18604.070440] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18604.248214] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18604.251739] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18604.253465] LNet: 1 peer NIs in recovery (showing 1): 10.240.22.189@tcp
[18604.295463] LNet: 843180:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18604.307105] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18604.312987] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18604.321151] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18604.388404] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18604.441663] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18604.447519] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18604.455825] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18604.524167] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.189@tcp
[18604.576993] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18604.582700] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18604.590724] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18607.545682] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18607.545682] lctl dl | grep ' ST ' || true
[18607.574017] LNet: 843401:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18607.575941] LNet: Removed LNI 10.240.22.180@tcp
[18607.594866] LNet: Removed LNI 10.240.22.180@tcp1
[18607.690892] Key type .llcrypt unregistered
[18607.691762] Key type ._llcrypt unregistered
[18611.502170] Key type ._llcrypt registered
[18611.504364] Key type .llcrypt registered
[18611.883506] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 04:41:15 \(1734410475\)
[18612.239444] Lustre: DEBUG MARKER: == sanity-lnet test 208: Test various kernel ip2nets configurations ========================================================== 04:41:15 (1734410475)
[18612.258899] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18612.258899] lctl dl | grep ' ST ' || true
[18612.322768] Key type .llcrypt unregistered
[18612.323639] Key type ._llcrypt unregistered
[18612.509623] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18612.511030] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18612.677993] Key type ._llcrypt registered
[18612.678897] Key type .llcrypt registered
[18612.743907] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18613.525692] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18613.527002] LNet: Accept all, port 7988
[18613.533970] LNet: 844194:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18613.546863] LNet: Removed LNI 10.240.22.180@tcp
[18613.560297] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18613.560297] lctl dl | grep ' ST ' || true
[18613.686778] Key type .llcrypt unregistered
[18613.687653] Key type ._llcrypt unregistered
[18613.895862] Key type ._llcrypt registered
[18613.896767] Key type .llcrypt registered
[18613.930388] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18614.705100] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18614.706312] LNet: Added LNI 10.1.2.3@tcp1 [8/256/0/180]
[18614.707548] LNet: Accept all, port 7988
[18614.714653] LNet: 844349:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18614.716384] LNet: Removed LNI 10.240.22.180@tcp
[18614.734770] LNet: Removed LNI 10.1.2.3@tcp1
[18614.748261] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18614.748261] lctl dl | grep ' ST ' || true
[18614.874775] Key type .llcrypt unregistered
[18614.875641] Key type ._llcrypt unregistered
[18615.050753] Key type ._llcrypt registered
[18615.051645] Key type .llcrypt registered
[18615.091087] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18615.866039] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18615.867258] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18615.868316] LNet: Accept all, port 7988
[18615.875272] LNet: 844504:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18615.876988] LNet: Removed LNI 10.240.22.180@tcp
[18615.894791] LNet: Removed LNI 10.1.2.3@tcp
[18615.908224] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18615.908224] lctl dl | grep ' ST ' || true
[18616.026792] Key type .llcrypt unregistered
[18616.027654] Key type ._llcrypt unregistered
[18616.223462] Key type ._llcrypt registered
[18616.224352] Key type .llcrypt registered
[18616.295229] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18617.077015] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18617.078197] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18617.079325] LNet: Accept all, port 7988
[18617.086212] LNet: 844660:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18617.087941] LNet: Removed LNI 10.240.22.180@tcp
[18617.094886] LNet: Removed LNI 10.1.2.3@tcp
[18617.108451] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18617.108451] lctl dl | grep ' ST ' || true
[18617.222822] Key type .llcrypt unregistered
[18617.223685] Key type ._llcrypt unregistered
[18617.438744] Key type ._llcrypt registered
[18617.439670] Key type .llcrypt registered
[18617.523405] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18618.325204] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18618.326394] LNetError: 844872:0:(config.c:1555:lnet_inet_select()) ksocklnd: failed to find interface test1pg
[18618.328207] LNetError: Error -100 starting up LNI tcp
[18618.342784] LNet: Removed LNI 10.240.22.180@tcp
[18618.356278] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18618.356278] lctl dl | grep ' ST ' || true
[18618.486771] Key type .llcrypt unregistered
[18618.487628] Key type ._llcrypt unregistered
[18620.146454] Key type ._llcrypt registered
[18620.147458] Key type .llcrypt registered
[18620.528081] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 04:41:24 \(1734410484\)
[18620.883029] Lustre: DEBUG MARKER: == sanity-lnet test 209: Check health, but not resends, for network timeout ========================================================== 04:41:24 (1734410484)
[18620.959406] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18620.959406] lctl dl | grep ' ST ' || true
[18621.018752] Key type .llcrypt unregistered
[18621.019620] Key type ._llcrypt unregistered
[18621.224976] Key type ._llcrypt registered
[18621.225902] Key type .llcrypt registered
[18621.311818] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18622.083934] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18622.088524] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18622.089652] LNet: Accept all, port 7988
[18625.758737] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18625.826716] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18625.830227] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18625.871335] LNet: 845775:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18625.882715] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18625.888276] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18625.897775] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18625.897775] lctl dl | grep ' ST ' || true
[18625.925867] LNet: 845803:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18625.950726] LNet: Removed LNI 10.240.22.180@tcp
[18626.042758] Key type .llcrypt unregistered
[18626.043626] Key type ._llcrypt unregistered
[18628.681628] Key type ._llcrypt registered
[18628.682572] Key type .llcrypt registered
[18628.753474] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18629.528972] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18629.533561] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18629.534763] LNet: Accept all, port 7988
[18633.193229] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18637.286165] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18637.291675] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18637.295242] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18637.471079] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[18637.474543] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18637.516423] LNet: 846470:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[18637.528077] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --health 1000 --all
[18637.533799] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 1000 --all
[18640.491231] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18640.506069] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18640.506069] lctl dl | grep ' ST ' || true
[18640.534409] LNet: 846535:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18640.536215] LNet: Removed LNI 10.240.22.180@tcp
[18640.558687] LNet: Removed LNI 10.240.22.180@tcp1
[18640.662630] Key type .llcrypt unregistered
[18640.663493] Key type ._llcrypt unregistered
[18644.478915] Key type ._llcrypt registered
[18644.481469] Key type .llcrypt registered
[18644.865679] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 210: Local NI recovery checks ======== 04:41:48 \(1734410508\)
[18645.222992] Lustre: DEBUG MARKER: == sanity-lnet test 210: Local NI recovery checks ======== 04:41:48 (1734410508)
[18645.281127] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18646.051006] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18646.060085] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18646.063901] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18646.065080] LNet: Accept all, port 7988
[18646.162026] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18646.165479] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18646.274505] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[18646.285616] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 10
[18646.302980] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 0 --nid 10.240.22.180@tcp
[18646.306753] LNetError: 847397:0:(api-ni.c:4061:lnet_ni_set_healthv()) manually adding local NI 10.240.22.180@tcp to recovery
[18646.312406] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18647.066582] LNet: 1 local NIs in recovery (showing 1): 10.240.22.180@tcp
[18647.066598] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18647.329877] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18648.348209] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18649.114831] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18649.367378] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18649.388514] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | 		 grep -wc nid-0: 10.240.22.180@tcp
[18649.400685] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[18649.407618] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18650.425356] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18651.443465] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18652.462475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18653.210803] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18653.481719] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18653.502932] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | 		 grep -wc nid-0: 10.240.22.180@tcp
[18653.515071] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[18653.527801] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18653.531241] LNet: 847524:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18653.532967] LNet: Removed LNI 10.240.22.180@tcp
[18653.546602] LNet: Removed LNI 10.240.22.180@tcp1
[18653.550000] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18653.558611] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18653.562369] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18653.563797] LNet: Accept all, port 7988
[18653.656259] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18653.659927] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18653.768193] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[18653.774460] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 0
[18653.784666] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 4
[18653.801983] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --health 0 --nid 10.240.22.180@tcp
[18653.805727] LNetError: 847785:0:(api-ni.c:4061:lnet_ni_set_healthv()) manually adding local NI 10.240.22.180@tcp to recovery
[18653.811479] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18654.828772] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18655.846393] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18656.602739] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18656.604482] LNet: Skipped 1 previous similar message
[18656.864648] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18656.885652] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | 		 grep -wc nid-0: 10.240.22.180@tcp
[18656.897753] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[18656.904375] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18657.922141] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18658.940045] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18659.957374] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18660.698516] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18660.974715] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18661.992730] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18663.011311] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18664.029652] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18665.048116] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v 2 | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18665.069008] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | 		 grep -wc nid-0: 10.240.22.180@tcp
[18665.081098] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[18665.088693] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 900
[18666.947442] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 211: Remote NI recovery checks ======= 04:42:10 \(1734410530\)
[18667.301140] Lustre: DEBUG MARKER: == sanity-lnet test 211: Remote NI recovery checks ======= 04:42:10 (1734410530)
[18667.316011] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18667.319604] LNet: 848397:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18668.346426] LNet: Removed LNI 10.240.22.180@tcp
[18668.362558] LNet: Removed LNI 10.240.22.180@tcp1
[18668.366620] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18668.375378] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18668.379223] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18668.380389] LNet: Accept all, port 7988
[18668.476660] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18668.480357] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18668.592100] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[18668.603047] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 10
[18668.618001] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --nid 10.240.22.180@tcp --health 0
[18668.625065] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18668.637140] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[18668.643935] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18669.402461] LNet: 1 peer NIs in recovery (showing 1): 10.240.22.180@tcp
[18669.403828] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18669.405450] LNet: Skipped 1 previous similar message
[18669.658966] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18670.674414] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18671.689714] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18672.705410] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18673.721234] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18674.736953] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18675.752573] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18676.768140] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18677.783515] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18678.798257] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18679.812585] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18680.826913] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18681.841189] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18682.855842] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18683.871061] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18683.883770] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[18683.890502] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18683.915237] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 0
[18683.920401] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --nid 10.240.22.180@tcp --health 500
[18683.927540] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18683.939608] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[18683.946195] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18684.960866] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18685.976044] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18686.990906] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18688.005303] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18689.019748] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18690.034159] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18690.046297] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[18690.058242] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18690.061609] LNet: 848907:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18690.063518] LNet: Removed LNI 10.240.22.180@tcp
[18690.074367] LNet: Removed LNI 10.240.22.180@tcp1
[18690.077758] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18690.086277] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18690.089965] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18690.091300] LNet: Accept all, port 7988
[18690.185777] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18690.189263] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18690.298559] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[18690.304851] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set recovery_limit 0
[18690.314829] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 4
[18690.329850] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --nid 10.240.22.180@tcp --health 0
[18690.337031] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18691.098318] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18691.100030] LNet: Skipped 3 previous similar messages
[18691.356215] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18691.379959] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18691.392211] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[18691.399043] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18692.417823] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18693.436753] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18694.457068] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18695.477549] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18696.497483] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18697.516590] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18698.535558] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18699.554453] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18700.573363] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18701.592247] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 2 --nid 10.240.22.180@tcp | 			 grep -v primary | 			 grep -e 10.240.22.180@tcp -e ping_count | grep -wA1 10.240.22.180@tcp | 			 awk '/ping_count/{print $NF}'
[18701.615830] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18701.628225] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[18701.636333] Lustre: DEBUG MARKER: /usr/sbin/lnetctl set max_recovery_ping_interval 900
[18703.492431] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 212: Check discovery refcount loss bug \(LU-14627\) ========================================================== 04:42:47 \(1734410567\)
[18703.859314] Lustre: DEBUG MARKER: == sanity-lnet test 212: Check discovery refcount loss bug (LU-14627) ========================================================== 04:42:47 (1734410567)
[18703.931768] LNet: 849847:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18703.933619] LNet: Removed LNI 10.240.22.180@tcp
[18703.942338] LNet: Removed LNI 10.240.22.180@tcp1
[18703.948770] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18703.948770] lctl dl | grep ' ST ' || true
[18704.074278] Key type .llcrypt unregistered
[18704.075159] Key type ._llcrypt unregistered
[18704.273598] Key type ._llcrypt registered
[18704.274512] Key type .llcrypt registered
[18704.339847] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18705.116057] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[18705.120482] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18705.121674] LNet: Accept all, port 7988
[18709.221799] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Initial discovery
[18709.576124] Lustre: DEBUG MARKER: Initial discovery
[18709.586401] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover --force 10.240.22.189@tcp
[18710.294946] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Fail local discover ping to set LNET_PEER_REDISCOVER flag
[18710.650342] Lustre: DEBUG MARKER: Fail local discover ping to set LNET_PEER_REDISCOVER flag
[18710.664798] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover --force 10.240.22.189@tcp
[18710.668522] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18711.071721] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Force onyx-138vm1.onyx.whamcloud.com to discover 10.240.22.189@tcp \(in background\)
[18711.426689] Lustre: DEBUG MARKER: Force onyx-138vm1.onyx.whamcloud.com to discover 10.240.22.189@tcp (in background)
[18711.439694] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover --force 10.240.22.189@tcp
[18711.832609] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Force onyx-138vm10 to discover 10.240.22.180@tcp
[18712.185984] Lustre: DEBUG MARKER: Force onyx-138vm10 to discover 10.240.22.180@tcp
[18712.894886] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Wait for 850439
[18713.251924] Lustre: DEBUG MARKER: Wait for 850439
[18714.682195] LNet: There was an unexpected network error while writing to 10.240.22.189: rc = -22
[18715.078138] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Finished wait on 850439
[18715.438018] Lustre: DEBUG MARKER: Finished wait on 850439
[18715.458075] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18715.458075] lctl dl | grep ' ST ' || true
[18715.486195] LNet: 850808:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18716.522217] LNet: Removed LNI 10.240.22.180@tcp
[18716.618185] Key type .llcrypt unregistered
[18716.619071] Key type ._llcrypt unregistered
[18720.431683] Key type ._llcrypt registered
[18720.432624] Key type .llcrypt registered
[18720.817088] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 04:43:04 \(1734410584\)
[18721.172803] Lustre: DEBUG MARKER: == sanity-lnet test 213: Check LNetDist calculation for multiple local NIDs ========================================================== 04:43:04 (1734410584)
[18721.192199] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18721.192199] lctl dl | grep ' ST ' || true
[18721.246150] Key type .llcrypt unregistered
[18721.247021] Key type ._llcrypt unregistered
[18721.445309] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[18721.446688] IPv6: ADDRCONF(NETDEV_CHANGE): test1pl: link becomes ready
[18721.541182] Key type ._llcrypt registered
[18721.542072] Key type .llcrypt registered
[18721.642641] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18722.413601] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18722.425670] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18722.429355] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18722.430878] LNet: Accept all, port 7988
[18722.526513] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[18722.529966] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18724.589770] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 04:43:08 \(1734410588\)
[18724.951012] Lustre: DEBUG MARKER: == sanity-lnet test 214: Check local NI status when link is downed ========================================================== 04:43:08 (1734410588)
[18724.969395] LNet: 852452:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18724.971302] LNet: Removed LNI 10.240.22.180@tcp
[18724.986265] LNet: Removed LNI 10.1.2.3@tcp
[18724.992778] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18724.992778] lctl dl | grep ' ST ' || true
[18725.122145] Key type .llcrypt unregistered
[18725.123037] Key type ._llcrypt unregistered
[18725.435237] Key type ._llcrypt registered
[18725.436164] Key type .llcrypt registered
[18725.497382] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18726.277863] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18726.286923] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18726.290680] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18726.291856] LNet: Accept all, port 7988
[18726.386193] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[18726.389711] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[18726.522260] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.1.2.3@tcp 10.240.22.180@tcp
[18727.546103] LNetError: Unexpected error -22 connecting to 10.240.22.180@tcp at host 10.240.22.180:7988
[18728.373828] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 04:43:11 \(1734410591\)
[18728.732123] Lustre: DEBUG MARKER: == sanity-lnet test 215: Test lnetctl ping --source option ========================================================== 04:43:11 (1734410591)
[18728.749219] LNet: 853357:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18728.751129] LNet: Removed LNI 10.240.22.180@tcp
[18728.766188] LNet: Removed LNI 10.1.2.3@tcp
[18728.772794] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18728.772794] lctl dl | grep ' ST ' || true
[18728.890178] Key type .llcrypt unregistered
[18728.891063] Key type ._llcrypt unregistered
[18729.102862] Key type ._llcrypt registered
[18729.103801] Key type .llcrypt registered
[18729.202214] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[18729.972639] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18729.981871] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18729.985537] LNet: 853508:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18729.987797] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18729.988968] LNet: Accept all, port 7988
[18730.083156] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp2 --if eth0
[18730.086854] LNet: Added LNI 10.240.22.180@tcp2 [8/256/0/180]
[18730.200725] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim 10.240.22.180@tcp1 --nid 10.240.22.180@tcp2
[18730.839775] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.22.180@tcp1 10.240.22.180@tcp2
[18730.843807] LNetError: 854067:0:(lib-move.c:2224:lnet_handle_find_routed_path()) Unable to send message from 10.240.22.180@tcp1 to 10.240.22.180@tcp2 - Route table may be misconfigured
[18730.848934] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.22.180@tcp2 10.240.22.180@tcp1
[18732.625548] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 04:43:16 \(1734410596\)
[18732.981878] Lustre: DEBUG MARKER: == sanity-lnet test 216: Failed send to peer NI owned by local host should not trigger peer NI recovery ========================================================== 04:43:16 (1734410596)
[18732.999185] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18733.002957] LNet: 854513:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18734.010036] LNet: Removed LNI 10.240.22.180@tcp1
[18735.050124] LNet: Removed LNI 10.240.22.180@tcp2
[18735.054176] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18735.063316] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18735.067257] LNet: 854520:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18735.069112] LNet: 854520:0:(lib-socket.c:567:lnet_inet_enumerate()) Skipped 1 previous similar message
[18735.071044] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18735.072162] LNet: Accept all, port 7988
[18735.167920] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18735.171589] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18735.281910] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[18735.288282] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.22.180@tcp 10.240.22.180@tcp
[18735.294594] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --source 10.240.22.180@tcp1 10.240.22.180@tcp1
[18735.309765] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[18735.313110] LNet: There was an unexpected network error while writing to 10.240.22.180: rc = -22
[18735.320634] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p | 		 grep -wc nid-0: 10.240.22.180@tcp
[18735.332617] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -p
[18735.339241] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l | 		 grep -wc nid-0: 10.240.22.180@tcp
[18735.351333] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug recovery -l
[18736.090051] LNet: 1 local NIs in recovery (showing 1): 10.240.22.180@tcp
[18737.092797] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 217: Don\'t leak memory when discovering peer with nnis \<= 1 ========================================================== 04:43:20 \(1734410600\)
[18737.446284] Lustre: DEBUG MARKER: == sanity-lnet test 217: Don't leak memory when discovering peer with nnis <= 1 ========================================================== 04:43:20 (1734410600)
[18737.463039] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18737.466593] LNet: 855236:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18738.490086] LNet: Removed LNI 10.240.22.180@tcp
[18739.530118] LNet: Removed LNI 10.240.22.180@tcp1
[18739.534475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18739.548159] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 0@lo
[18739.556636] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[18739.556636] lctl dl | grep ' ST ' || true
[18739.585033] LNet: 855266:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18739.678165] Key type .llcrypt unregistered
[18739.679042] Key type ._llcrypt unregistered
[18741.332528] Key type ._llcrypt registered
[18741.333468] Key type .llcrypt registered
[18741.716947] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 04:43:25 \(1734410605\)
[18742.075139] Lustre: DEBUG MARKER: == sanity-lnet test 218: Local recovery pings should exercise all available paths ========================================================== 04:43:25 (1734410605)
[18742.132650] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18742.900576] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18743.301465] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_218 Need two LNet interfaces
[18743.664471] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_218 Need two LNet interfaces
[18744.088092] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 219: Consolidate peer entries ======== 04:43:27 \(1734410607\)
[18744.444908] Lustre: DEBUG MARKER: == sanity-lnet test 219: Consolidate peer entries ======== 04:43:27 (1734410607)
[18744.458830] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18744.462410] LNet: 856081:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18744.466853] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[18744.475626] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18744.479329] LNet: 856088:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18744.481256] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18744.482474] LNet: Accept all, port 7988
[18744.577998] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[18744.581419] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[18744.695310] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[18744.701388] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp1
[18744.707484] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp1
[18746.463287] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 220: Add routes w\/default options - check aliveness ========================================================== 04:43:30 \(1734410610\)
[18746.826472] Lustre: DEBUG MARKER: == sanity-lnet test 220: Add routes w/default options - check aliveness ========================================================== 04:43:30 (1734410610)
[18755.944045] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18756.307588] LNet: 857181:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18756.309517] LNet: Removed LNI 10.240.22.180@tcp
[18756.325983] LNet: Removed LNI 10.240.22.180@tcp1
[18756.425976] Key type .llcrypt unregistered
[18756.426835] Key type ._llcrypt unregistered
[18758.576420] Key type ._llcrypt registered
[18758.577335] Key type .llcrypt registered
[18758.950271] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18759.464835] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18759.832921] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18761.106222] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18767.546816] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18767.550881] LNet: 857755:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18767.552845] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18767.554143] LNet: Accept all, port 7988
[18768.914375] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18768.922935] LNet: 857914:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18769.577080] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18769.600502] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18770.903300] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18771.862798] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/us
[18773.111963] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_module ..\/lnet\/selftest\/lnet_selftest
[18773.574678] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_module ../lnet/selftest/lnet_selftest
[18774.962338] LNet: 858277:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18775.066069] LNet: 858277:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18775.170176] LNet: 858277:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18775.274128] LNet: 858277:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18775.378109] LNet: 858277:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18775.482102] LNet: 858277:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18775.586131] LNet: 858277:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18777.040594] Lustre: DEBUG MARKER: Start LST rw
[18777.045444] Lustre: DEBUG MARKER: Start LST rw
[18777.058977] LNet: 821893:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18777.164747] LNet: 796305:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18783.336257] Autotest: Test running for 310 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[18792.286985] Lustre: DEBUG MARKER: lst stop brw_rw
[18792.314352] Lustre: DEBUG MARKER: lst stop brw_rw
[18793.326016] Lustre: DEBUG MARKER: Stop LST rw
[18793.343611] Lustre: DEBUG MARKER: Stop LST rw
[18793.416853] Lustre: DEBUG MARKER: Start LST rw
[18793.421392] Lustre: DEBUG MARKER: Start LST rw
[18793.434768] LNet: 782712:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18793.553569] LNet: 844840:0:(rpc.c:645:srpc_service_add_buffers()) waiting for adding buffer
[18808.709028] Lustre: DEBUG MARKER: lst stop brw_rw
[18808.728638] Lustre: DEBUG MARKER: lst stop brw_rw
[18809.749362] Lustre: DEBUG MARKER: Stop LST rw
[18809.766533] Lustre: DEBUG MARKER: Stop LST rw
[18809.785977] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18810.803550] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18811.199273] LNet: 858633:0:(timer.c:220:stt_shutdown()) waiting for 1 threads to terminate
[18811.357707] LNet: 858633:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18812.393645] LNet: Removed LNI 10.240.22.180@tcp
[18812.498041] Key type .llcrypt unregistered
[18812.498922] Key type ._llcrypt unregistered
[18813.936298] Key type ._llcrypt registered
[18813.937571] Key type .llcrypt registered
[18814.321547] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 221: Add routes w\/DD disabled - check aliveness ========================================================== 04:44:37 \(1734410677\)
[18814.677474] Lustre: DEBUG MARKER: == sanity-lnet test 221: Add routes w/DD disabled - check aliveness ========================================================== 04:44:37 (1734410677)
[18823.797786] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18824.205531] Key type .llcrypt unregistered
[18824.206464] Key type ._llcrypt unregistered
[18826.417242] Key type ._llcrypt registered
[18826.420090] Key type .llcrypt registered
[18826.797636] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18827.374624] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18827.690219] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18834.046057] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18840.474162] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18840.478220] LNet: 860071:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18840.480180] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18840.481315] LNet: Accept all, port 7988
[18841.841743] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18841.850371] LNet: 860230:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
[18841.852610] LNet: 860230:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18842.502939] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18842.520881] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18843.822738] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18844.468939] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18845.478698] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18845.848755] LNet: 860410:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18845.869436] LNet: Removed LNI 10.240.22.180@tcp
[18845.961466] Key type .llcrypt unregistered
[18845.962337] Key type ._llcrypt unregistered
[18848.176077] Key type ._llcrypt registered
[18848.176981] Key type .llcrypt registered
[18848.571295] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 04:45:12 \(1734410712\)
[18848.945589] Lustre: DEBUG MARKER: == sanity-lnet test 222: Check avoid_asym_router_failure=1 ========================================================== 04:45:12 (1734410712)
[18858.095233] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18858.497346] Key type .llcrypt unregistered
[18858.498233] Key type ._llcrypt unregistered
[18865.692906] Key type ._llcrypt registered
[18865.693867] Key type .llcrypt registered
[18866.074340] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18866.699806] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18867.122550] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18868.344883] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18874.767278] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18874.771342] LNet: 861849:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18874.773285] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18874.774440] LNet: Accept all, port 7988
[18883.184358] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18883.188079] LNetError: 862132:0:(lib-move.c:2284:lnet_handle_find_routed_path()) peer 10.240.22.181@tcp1 has no available nets
[18883.832812] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18883.841721] LNet: 862146:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18883.846520] LNetError: 861736:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from up to down
[18884.471045] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18884.489202] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18884.503379] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18884.506791] LNetError: 862197:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.22.181@tcp1 from <?>
[18885.768745] LNetError: 861736:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from down to up
[18886.396209] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18886.414349] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18887.703649] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18888.344540] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18888.348066] LNet: 862308:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18888.365196] LNet: Removed LNI 10.240.22.180@tcp
[18900.025942] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18900.675025] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18901.676815] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18902.153142] Key type .llcrypt unregistered
[18902.154043] Key type ._llcrypt unregistered
[18904.367693] Key type ._llcrypt registered
[18904.368582] Key type .llcrypt registered
[18904.754419] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 223: Check avoid_asym_router_failure=1 w\/DD disabled ========================================================== 04:46:08 \(1734410768\)
[18905.115483] Lustre: DEBUG MARKER: == sanity-lnet test 223: Check avoid_asym_router_failure=1 w/DD disabled ========================================================== 04:46:08 (1734410768)
[18914.177279] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18914.573012] Key type .llcrypt unregistered
[18914.573965] Key type ._llcrypt unregistered
[18916.685962] Key type ._llcrypt registered
[18916.686851] Key type .llcrypt registered
[18917.059362] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18917.556999] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=1 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18917.848029] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18919.321403] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18925.699500] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18925.703371] LNet: 864070:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18925.705440] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18925.706610] LNet: Accept all, port 7988
[18934.143173] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18934.146926] LNetError: 864352:0:(lib-move.c:2284:lnet_handle_find_routed_path()) peer 10.240.22.181@tcp1 has no available nets
[18934.793918] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18934.802769] LNet: 864366:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
[18934.805022] LNet: 864366:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[18934.808035] LNetError: 863957:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from up to down
[18935.448362] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18935.466025] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18935.479707] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18935.483123] LNetError: 864417:0:(lib-move.c:2324:lnet_handle_find_routed_path()) no route to 10.240.22.181@tcp1 from <?>
[18937.397293] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18937.414417] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18938.433260] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18938.450437] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18939.468974] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18939.486406] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18939.929189] LNetError: 863957:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to tcp1 through 10.240.22.189@tcp has gone from down to up
[18940.505312] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18940.522542] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18941.819528] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18942.458031] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18942.461613] LNet: 864597:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18942.476906] LNet: Removed LNI 10.240.22.180@tcp
[18957.921826] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18958.571585] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[18959.573753] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18960.024775] Key type .llcrypt unregistered
[18960.025684] Key type ._llcrypt unregistered
[18962.291925] Key type ._llcrypt registered
[18962.292933] Key type .llcrypt registered
[18962.677972] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 04:47:06 \(1734410826\)
[18963.035798] Lustre: DEBUG MARKER: == sanity-lnet test 224: Check avoid_asym_router_failure=0 ========================================================== 04:47:06 (1734410826)
[18972.123410] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[18972.524646] Key type .llcrypt unregistered
[18972.525557] Key type ._llcrypt unregistered
[18974.669786] Key type ._llcrypt registered
[18974.670697] Key type .llcrypt registered
[18975.046845] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18975.649935] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[18976.015248] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[18977.320076] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[18983.755850] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[18983.759754] LNet: 866415:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[18983.761951] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[18983.763113] LNet: Accept all, port 7988
[18992.186405] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[18992.819198] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18992.836874] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18994.113057] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[18994.131304] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[18995.439234] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[18996.078506] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[18996.082015] LNet: 866851:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[18996.096564] LNet: Removed LNI 10.240.22.180@tcp
[19007.682301] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[19008.680884] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19009.136472] Key type .llcrypt unregistered
[19009.137400] Key type ._llcrypt unregistered
[19011.354537] Key type ._llcrypt registered
[19011.358728] Key type .llcrypt registered
[19011.741556] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 225: Check avoid_asym_router_failure=0 w\/DD disabled ========================================================== 04:47:55 \(1734410875\)
[19012.098181] Lustre: DEBUG MARKER: == sanity-lnet test 225: Check avoid_asym_router_failure=0 w/DD disabled ========================================================== 04:47:55 (1734410875)
[19021.135861] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19021.532334] Key type .llcrypt unregistered
[19021.533258] Key type ._llcrypt unregistered
[19023.721830] Key type ._llcrypt registered
[19023.722758] Key type .llcrypt registered
[19024.095397] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[19024.685092] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet avoid_asym_router_failure=0 lnet_peer_discovery_disabled=1 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[19025.092418] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19026.340359] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[19032.716553] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19032.720598] LNet: 868604:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19032.722551] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19032.723693] LNet: Accept all, port 7988
[19041.108912] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[19041.117660] LNet: 868889:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
[19041.748104] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[19041.765788] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[19043.033881] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[19043.051983] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[19044.343900] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[19044.983769] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19044.987304] LNet: 869040:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19045.008216] LNet: Removed LNI 10.240.22.180@tcp
[19056.586574] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[19057.592670] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19058.056122] Key type .llcrypt unregistered
[19058.059833] Key type ._llcrypt unregistered
[19060.349718] Key type ._llcrypt registered
[19060.350673] Key type .llcrypt registered
[19060.736411] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 04:48:44 \(1734410924\)
[19061.094577] Lustre: DEBUG MARKER: == sanity-lnet test 226: test missing route for 1 of 2 routers ========================================================== 04:48:44 (1734410924)
[19073.405995] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19073.812017] Key type .llcrypt unregistered
[19073.812943] Key type ._llcrypt unregistered
[19076.144129] Key type ._llcrypt registered
[19076.145021] Key type .llcrypt registered
[19076.519612] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[19077.235636] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[19077.770363] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19079.137571] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[19084.361759] Autotest: Test running for 315 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[19089.624912] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19089.632970] LNet: 870988:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19089.634904] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19089.636183] LNet: Accept all, port 7988
[19091.954710] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[19091.963807] LNet: 871169:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[19091.970006] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.181@tcp
[19092.605624] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[19092.624434] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[19093.272420] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.592991] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.599226] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.605397] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.611512] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.964341] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.973675] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.979741] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.985964] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19093.999940] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[19094.017320] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.181@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.181@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.181@tcp;			e
[19095.049167] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19095.415321] LNet: 871463:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19095.431923] LNet: Removed LNI 10.240.22.180@tcp
[19095.527905] Key type .llcrypt unregistered
[19095.528816] Key type ._llcrypt unregistered
[19097.790200] Key type ._llcrypt registered
[19097.791142] Key type .llcrypt registered
[19098.173946] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 227: Check router peer health w\/DD disabled ========================================================== 04:49:21 \(1734410961\)
[19098.536896] Lustre: DEBUG MARKER: == sanity-lnet test 227: Check router peer health w/DD disabled ========================================================== 04:49:21 (1734410961)
[19109.894484] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19110.291846] Key type .llcrypt unregistered
[19110.292822] Key type ._llcrypt unregistered
[19112.604411] Key type ._llcrypt registered
[19112.605313] Key type .llcrypt registered
[19112.979956] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 
[19113.462708] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet lnet_peer_discovery_disabled=1 lnet_health_sensitivity=0 lnet_transaction_timeout=10 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=
[19113.858311] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19115.570922] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[19124.048381] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19124.052515] LNet: 872988:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19124.054469] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19124.055599] LNet: Accept all, port 7988
[19125.726602] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[19125.735480] LNet: 873160:0:(router.c:722:lnet_add_route()) Consider turning discovery on to enable full Multi-Rail routing functionality
[19125.737727] LNet: 873160:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[19126.707603] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[19126.726023] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[19128.671115] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.182@tcp1
[19128.677593] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[19130.294109] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[19141.079632] LNet: 1 peer NIs in recovery (showing 1): 10.240.22.189@tcp
[19141.084250] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[19151.702754] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[19152.454341] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19152.817251] LNet: 873423:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19152.835583] LNet: Removed LNI 10.240.22.180@tcp
[19152.931552] Key type .llcrypt unregistered
[19152.932418] Key type ._llcrypt unregistered
[19155.166556] Key type ._llcrypt registered
[19155.167533] Key type .llcrypt registered
[19155.547817] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 230: Test setting conns-per-peer ===== 04:50:19 \(1734411019\)
[19155.911650] Lustre: DEBUG MARKER: == sanity-lnet test 230: Test setting conns-per-peer ===== 04:50:19 (1734411019)
[19155.974294] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19156.746152] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19156.755254] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19156.759026] LNet: 873937:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19156.761254] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19156.762398] LNet: Accept all, port 7988
[19156.856961] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 4
[19156.870801] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19156.878584] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19156.898371] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19156.901655] LNet: 874074:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19156.923551] LNet: Removed LNI 10.240.22.180@tcp
[19156.926944] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19156.935713] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19156.939619] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19156.940784] LNet: Accept all, port 7988
[19157.036911] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 5
[19157.050925] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19157.058522] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19157.077278] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19157.099570] LNet: Removed LNI 10.240.22.180@tcp
[19157.102923] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19157.111543] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19157.115302] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19157.116673] LNet: Accept all, port 7988
[19157.210094] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 6
[19157.223626] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19157.230968] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19157.249583] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19157.271562] LNet: Removed LNI 10.240.22.180@tcp
[19157.274885] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19157.283380] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19157.286990] LNet: 874369:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19157.288858] LNet: 874369:0:(lib-socket.c:567:lnet_inet_enumerate()) Skipped 2 previous similar messages
[19157.290711] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19157.291930] LNet: Accept all, port 7988
[19157.384744] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 7
[19157.398259] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19157.405921] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19157.424464] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19157.427862] LNet: 874506:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19157.429504] LNet: 874506:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 2 previous similar messages
[19158.471527] LNet: Removed LNI 10.240.22.180@tcp
[19158.475217] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19158.483936] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19158.487629] LNet: 874514:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19158.489562] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19158.490699] LNet: Accept all, port 7988
[19158.582714] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 8
[19158.596462] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19158.603777] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19158.622807] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19158.626082] LNet: 874651:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19158.639521] LNet: Removed LNI 10.240.22.180@tcp
[19158.642854] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19158.651309] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19158.654945] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19158.656310] LNet: Accept all, port 7988
[19158.748804] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 9
[19158.762567] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19158.770348] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19158.789003] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19158.811584] LNet: Removed LNI 10.240.22.180@tcp
[19158.814944] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19158.823420] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19158.827127] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19158.828492] LNet: Accept all, port 7988
[19158.922289] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 10
[19158.935748] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19158.943111] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19158.961787] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19158.983545] LNet: Removed LNI 10.240.22.180@tcp
[19158.986814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19158.995268] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19158.999045] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19159.000412] LNet: Accept all, port 7988
[19159.098848] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 11
[19159.112433] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19159.120779] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19159.139543] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19159.155617] LNet: Removed LNI 10.240.22.180@tcp
[19159.158912] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19159.167463] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19159.171215] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19159.172548] LNet: Accept all, port 7988
[19159.266883] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 12
[19159.280345] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19159.288662] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19159.307236] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19159.327559] LNet: Removed LNI 10.240.22.180@tcp
[19159.330834] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19159.339291] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19159.343047] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19159.344421] LNet: Accept all, port 7988
[19159.438728] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 13
[19159.452321] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19159.460675] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19159.479384] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19159.499570] LNet: Removed LNI 10.240.22.180@tcp
[19159.502895] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19159.511422] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19159.515190] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19159.516544] LNet: Accept all, port 7988
[19159.610334] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 14
[19159.624030] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19159.631574] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19159.650814] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19159.667605] LNet: Removed LNI 10.240.22.180@tcp
[19159.670971] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19159.679566] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19159.683321] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19159.684663] LNet: Accept all, port 7988
[19159.778322] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 15
[19159.791846] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19159.800752] Lustre: DEBUG MARKER: printf 'network tcp\nconn_list\n' | lctl | grep -c '10.240.22.180@tcp'
[19159.819720] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19159.835624] LNet: Removed LNI 10.240.22.180@tcp
[19159.838934] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19159.847489] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19159.851243] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19159.852617] LNet: Accept all, port 7988
[19159.946868] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer 128
[19159.957586] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19159.979553] LNet: Removed LNI 10.240.22.180@tcp
[19159.982833] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19159.991488] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19159.995153] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19159.996485] LNet: Accept all, port 7988
[19160.095537] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net set --all --conns-per-peer -1
[19161.856332] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 04:50:25 \(1734411025\)
[19162.213752] Lustre: DEBUG MARKER: == sanity-lnet test 231: Check DLC handling of peer_timeout parameter ========================================================== 04:50:25 (1734411025)
[19162.231192] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19162.234617] LNet: 876360:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19162.236291] LNet: 876360:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 8 previous similar messages
[19162.259551] LNet: Removed LNI 10.240.22.180@tcp
[19162.262916] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19162.271560] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp231 --if eth0
[19162.275229] LNet: 876367:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19162.277122] LNet: 876367:0:(lib-socket.c:567:lnet_inet_enumerate()) Skipped 9 previous similar messages
[19162.279039] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/180]
[19162.280361] LNet: Accept all, port 7988
[19162.451031] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19162.467602] LNet: Removed LNI 10.240.22.180@tcp231
[19162.470918] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19162.481288] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/sanity-lnet-231-expected.yaml
[19162.485336] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/180]
[19162.486540] LNet: Accept all, port 7988
[19162.487696] LNet: 876502:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[19162.551642] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp231 --if eth0
[19162.567578] LNet: Removed LNI 10.240.22.180@tcp231
[19162.570601] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp231 --if eth0 --peer-timeout=-1
[19162.574695] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/180]
[19162.730042] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19162.747570] LNet: Removed LNI 10.240.22.180@tcp231
[19162.750880] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19162.761120] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/sanity-lnet-231-expected.yaml
[19162.765142] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/0]
[19162.766325] LNet: Accept all, port 7988
[19162.826989] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp231 --if eth0
[19162.843548] LNet: Removed LNI 10.240.22.180@tcp231
[19162.846577] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp231 --if eth0 --peer-timeout=0
[19162.850641] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/0]
[19163.007638] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19163.023593] LNet: Removed LNI 10.240.22.180@tcp231
[19163.026932] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19163.037205] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/sanity-lnet-231-expected.yaml
[19163.041280] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/60]
[19163.042467] LNet: Accept all, port 7988
[19163.043409] LNet: 876795:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[19163.045254] LNet: 876795:0:(api-ni.c:337:recovery_interval_set()) Skipped 1 previous similar message
[19163.106194] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp231 --if eth0
[19163.127516] LNet: Removed LNI 10.240.22.180@tcp231
[19163.130565] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp231 --if eth0 --peer-timeout=60
[19163.134681] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/60]
[19163.290299] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19163.307572] LNet: Removed LNI 10.240.22.180@tcp231
[19163.310905] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19163.321148] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import /tmp/sanity-lnet-231-expected.yaml
[19163.325217] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/180]
[19163.326436] LNet: Accept all, port 7988
[19163.386549] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp231 --if eth0
[19163.403542] LNet: Removed LNI 10.240.22.180@tcp231
[19163.406561] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp231 --if eth0 --peer-timeout=180
[19163.410664] LNet: Added LNI 10.240.22.180@tcp231 [8/256/0/180]
[19165.310226] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 232: Test setting ToS value ========== 04:50:28 \(1734411028\)
[19165.666879] Lustre: DEBUG MARKER: == sanity-lnet test 232: Test setting ToS value ========== 04:50:28 (1734411028)
[19165.682205] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19165.703547] LNet: Removed LNI 10.240.22.180@tcp231
[19165.706948] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19165.718238] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp232 --if eth0
[19165.722071] LNet: Added LNI 10.240.22.180@tcp232 [8/256/0/180]
[19165.723290] LNet: Accept all, port 7988
[19165.816963] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show --net tcp232 -v 1
[19165.823168] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp232 --if eth0
[19165.847537] LNet: Removed LNI 10.240.22.180@tcp232
[19165.850627] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp232 --if eth0 --tos -1
[19165.854650] LNet: Added LNI 10.240.22.180@tcp232 [8/256/0/180]
[19165.949577] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show --net tcp232 -v 1
[19165.956016] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp232 --if eth0
[19165.979526] LNet: Removed LNI 10.240.22.180@tcp232
[19165.982625] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp232 --if eth0 --tos 104
[19165.986626] LNet: Added LNI 10.240.22.180@tcp232 [8/256/0/180]
[19166.079132] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show --net tcp232 -v 1
[19166.085341] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp232 --if eth0
[19166.111539] LNet: Removed LNI 10.240.22.180@tcp232
[19166.114594] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp232 --if eth0 --tos 106
[19166.118611] LNet: Added LNI 10.240.22.180@tcp232 [8/256/0/180]
[19166.212475] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show --net tcp232 -v 1
[19166.218573] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp232 --if eth0
[19166.231594] LNet: Removed LNI 10.240.22.180@tcp232
[19166.234624] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp232 --if eth0 --tos 300
[19167.988035] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 250: test that linux routes are added ========================================================== 04:50:31 \(1734411031\)
[19168.344868] Lustre: DEBUG MARKER: == sanity-lnet test 250: test that linux routes are added ========================================================== 04:50:31 (1734411031)
[19168.363128] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19168.366524] LNet: 878460:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19168.368194] LNet: 878460:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Skipped 5 previous similar messages
[19168.372583] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19168.381106] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19168.384744] LNet: 878467:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19168.386613] LNet: 878467:0:(lib-socket.c:567:lnet_inet_enumerate()) Skipped 12 previous similar messages
[19168.388620] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19168.389753] LNet: Accept all, port 7988
[19170.242706] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 04:50:33 \(1734411033\)
[19170.597692] Lustre: DEBUG MARKER: == sanity-lnet test 251: Define multiple kfi networks on single interface ========================================================== 04:50:33 (1734411033)
[19170.997157] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_251 Need kfi NETTYPE
[19171.351907] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_251 Need kfi NETTYPE
[19171.780865] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 04:50:35 \(1734411035\)
[19172.141765] Lustre: DEBUG MARKER: == sanity-lnet test 252: Ping to down peer should unlink quickly ========================================================== 04:50:35 (1734411035)
[19172.231530] LNet: Removed LNI 10.240.22.180@tcp
[19172.237996] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19172.237996] lctl dl | grep ' ST ' || true
[19172.355628] Key type .llcrypt unregistered
[19172.356501] Key type ._llcrypt unregistered
[19172.556686] Key type ._llcrypt registered
[19172.557572] Key type .llcrypt registered
[19172.611967] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19173.394786] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[19173.399031] LNet: 879471:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19173.401249] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19173.402391] LNet: Accept all, port 7988
[19177.104005] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[19180.386227] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping --timeout 15 10.240.22.189@tcp
[19180.403241] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19180.403241] lctl dl | grep ' ST ' || true
[19180.432861] LNet: 879634:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19180.447458] LNet: Removed LNI 10.240.22.180@tcp
[19180.547445] Key type .llcrypt unregistered
[19180.548323] Key type ._llcrypt unregistered
[19182.257444] Key type ._llcrypt registered
[19182.260205] Key type .llcrypt registered
[19182.647114] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253
[19183.004289] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_253 skipping ALWAYS excluded test 253
[19183.415420] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254
[19183.768965] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_254 skipping ALWAYS excluded test 254
[19184.189539] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 04:50:47 \(1734411047\)
[19184.546644] Lustre: DEBUG MARKER: == sanity-lnet test 255: Use lnet routes param with pdsh syntax ========================================================== 04:50:47 (1734411047)
[19184.605756] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19185.375437] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19185.390668] LNet: 880439:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19185.398661] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19185.398661] lctl dl | grep ' ST ' || true
[19185.527414] Key type .llcrypt unregistered
[19185.528279] Key type ._llcrypt unregistered
[19185.745627] Key type ._llcrypt registered
[19185.746576] Key type .llcrypt registered
[19185.800817] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19186.579001] LNet: 880577:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19186.581102] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19186.582239] LNet: 880577:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[19186.584959] LNet: Accept all, port 7988
[19186.586390] LNetError: 880578:0:(lib-lnet.h:1328:lnet_set_route_aliveness()) route to o2ib through 10.240.22.181@tcp has gone from up to down
[19186.589461] LNetError: 880587:0:(router.c:399:lnet_router_discovery_ping_reply()) Peer 10.240.22.183@tcp is being used as a gateway but routing feature is not turned on
[19186.621843] LNetError: 880605:0:(router.c:333:lnet_is_route_alive()) peer 10.240.22.185@tcp is being used as a gateway but routing feature is not turned on
[19186.669119] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19186.672674] LNet: 880630:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19186.687402] LNet: Removed LNI 10.240.22.180@tcp
[19186.690932] Lustre: DEBUG MARKER: /usr/sbin/lnetctl import
[19186.694311] LNet: 880632:0:(api-ni.c:337:recovery_interval_set()) 'lnet_recovery_interval' has been deprecated
[19188.465756] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 04:50:52 \(1734411052\)
[19188.821011] Lustre: DEBUG MARKER: == sanity-lnet test 256: Router should not drop messages that are past the deadline ========================================================== 04:50:52 (1734411052)
[19197.889839] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19198.343328] Key type .llcrypt unregistered
[19198.344235] Key type ._llcrypt unregistered
[19200.399581] Key type ._llcrypt registered
[19200.400551] Key type .llcrypt registered
[19200.775711] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[19201.191834] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_lnet peer_buffer_credits=1024 alive_router_check_interval=5 router_ping_timeout=5 large_router_buffers=4 small_router_buffers=8 tiny_router_buffers=9
[19201.497638] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19203.077893] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure 
[19209.472935] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19209.478612] LNet: 882053:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19209.480814] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19209.481989] LNet: Accept all, port 7988
[19210.847377] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route add --net tcp1 --gateway 10.240.22.189@tcp
[19210.856092] LNet: 882212:0:(router.c:737:lnet_add_route()) Use hops = 1 for a single-hop route when avoid_asym_router_failure feature is enabled
[19211.507702] Lustre: DEBUG MARKER: /usr/sbin/lctl show_route
[19211.526123] Lustre: DEBUG MARKER: /usr/sbin/lnetctl route show -v
[19212.822712] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.181@tcp1
[19227.935304] Lustre: DEBUG MARKER: output="$(/usr/sbin/lnetctl route show --net tcp1 --gateway 10.240.22.189@tcp 2>/dev/null)"; 		if [[ -n "${output}" ]]; then						echo "Delete route to tcp1 via 10.240.22.189@tcp";				/usr/sbin/lnetctl route del --net tcp1 --gateway 10.240.22.189@tcp;			e
[19228.959981] Lustre: DEBUG MARKER: /usr/sbin/lustre_rmmod
[19229.324525] LNet: 882531:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19229.343118] LNet: Removed LNI 10.240.22.180@tcp
[19229.443100] Key type .llcrypt unregistered
[19229.443976] Key type ._llcrypt unregistered
[19231.659118] Key type ._llcrypt registered
[19231.660048] Key type .llcrypt registered
[19232.042317] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 04:51:35 \(1734411095\)
[19232.401288] Lustre: DEBUG MARKER: == sanity-lnet test 260: test that linux sysctl parameter are set correctly ========================================================== 04:51:35 (1734411095)
[19232.540274] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19233.351074] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[19233.406027] LNet: 883120:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19233.408038] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19233.409206] LNet: Accept all, port 7988
[19234.947050] Key type lgssc registered
[19235.132439] Lustre: Echo OBD driver; http://www.lustre.org/
[19235.271489] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19235.271489] lctl dl | grep ' ST ' || true
[19235.666939] Key type lgssc unregistered
[19235.739106] LNet: 883379:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19235.759108] LNet: Removed LNI 10.240.22.180@tcp
[19235.971115] Key type .llcrypt unregistered
[19235.972074] Key type ._llcrypt unregistered
[19237.662718] Key type ._llcrypt registered
[19237.663729] Key type .llcrypt registered
[19238.043108] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 04:51:41 \(1734411101\)
[19238.401202] Lustre: DEBUG MARKER: == sanity-lnet test 300: packaged LNet UAPI headers can be compiled ========================================================== 04:51:41 (1734411101)
[19238.426260] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19238.426260] lctl dl | grep ' ST ' || true
[19238.491054] Key type .llcrypt unregistered
[19238.491920] Key type ._llcrypt unregistered
[19238.690263] Key type ._llcrypt registered
[19238.692350] Key type .llcrypt registered
[19238.752555] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19241.871403] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 301: Check for dynamic adds of same\/wrong interface \(memory leak\) ========================================================== 04:51:45 \(1734411105\)
[19242.228296] Lustre: DEBUG MARKER: == sanity-lnet test 301: Check for dynamic adds of same/wrong interface (memory leak) ========================================================== 04:51:45 (1734411105)
[19242.245806] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19242.251227] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19242.260266] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19242.264051] LNet: 884550:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19242.265992] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19242.267405] LNet: Accept all, port 7988
[19242.359991] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19242.365790] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net del --net tcp --if eth0
[19242.391075] LNet: Removed LNI 10.240.22.180@tcp
[19242.397149] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19242.397149] lctl dl | grep ' ST ' || true
[19242.425182] LNet: 884689:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19242.523082] Key type .llcrypt unregistered
[19242.523950] Key type ._llcrypt unregistered
[19244.195838] Key type ._llcrypt registered
[19244.196769] Key type .llcrypt registered
[19244.578142] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 04:51:48 \(1734411108\)
[19244.933630] Lustre: DEBUG MARKER: == sanity-lnet test 302: Check that peer debug info can be dumped ========================================================== 04:51:48 (1734411108)
[19244.993357] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19245.765124] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19245.776354] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19245.780295] LNet: 885260:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19245.782255] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19245.783555] LNet: Accept all, port 7988
[19245.885196] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19245.891752] Lustre: DEBUG MARKER: /usr/sbin/lnetctl debug peer --nid 10.240.22.180@tcp
[19245.895316] LNet: Conn 0 [type, closing, crefcnt, srefcnt]: 2, 0, 1, 1
[19245.896582] LNet: Conn 0 rx [scheduled, ready, state]: 0, 0, 1
[19245.897704] LNet: Conn 0 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 19245, 0, 0
[19245.899329] LNet: Conn 0 sched [nconns, cpt]: 4, 0
[19245.900273] LNet: Conn 1 [type, closing, crefcnt, srefcnt]: 3, 0, 1, 1
[19245.901507] LNet: Conn 1 rx [scheduled, ready, state]: 0, 0, 1
[19245.902619] LNet: Conn 1 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 19245, 0, 0
[19245.904213] LNet: Conn 1 sched [nconns, cpt]: 4, 0
[19245.905152] LNet: Conn 2 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1
[19245.906375] LNet: Conn 2 rx [scheduled, ready, state]: 0, 0, 1
[19245.907490] LNet: Conn 2 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 19245, 0, 19294
[19245.909136] LNet: Conn 2 sched [nconns, cpt]: 4, 0
[19245.910074] LNet: Conn 3 [type, closing, crefcnt, srefcnt]: 1, 0, 1, 1
[19245.911320] LNet: Conn 3 rx [scheduled, ready, state]: 0, 0, 1
[19245.912450] LNet: Conn 3 tx [txqcnt, scheduled, last_post, ready, deadline]: 0, 0, 19245, 0, 19294
[19245.914096] LNet: Conn 3 sched [nconns, cpt]: 4, 0
[19247.687262] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 303: Check peer NI health after link down ========================================================== 04:51:51 \(1734411111\)
[19248.043485] Lustre: DEBUG MARKER: == sanity-lnet test 303: Check peer NI health after link down ========================================================== 04:51:51 (1734411111)
[19248.123912] LNet: 885877:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19248.147112] LNet: Removed LNI 10.240.22.180@tcp
[19248.153906] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19248.153906] lctl dl | grep ' ST ' || true
[19248.267063] Key type .llcrypt unregistered
[19248.267961] Key type ._llcrypt unregistered
[19248.489406] Key type ._llcrypt registered
[19248.490343] Key type .llcrypt registered
[19248.532676] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19249.302197] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[19249.306553] LNet: 886022:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19249.308779] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19249.309932] LNet: Accept all, port 7988
[19253.014799] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.189@tcp
[19257.187822] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19257.193461] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp1 --if eth0
[19257.197011] LNet: 886202:0:(lib-socket.c:567:lnet_inet_enumerate()) lnet: Ignoring interface test1pg: it's down
[19257.198929] LNet: Added LNI 10.240.22.180@tcp1 [8/256/0/180]
[19257.322844] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[19257.343672] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp99 --if test1pg
[19257.347709] LNet: Added LNI 10.1.2.3@tcp99 [8/256/0/180]
[19260.094180] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 04:52:03 \(1734411123\)
[19260.450958] Lustre: DEBUG MARKER: == sanity-lnet test 304: Check locked primary peer nid consolidation ========================================================== 04:52:03 (1734411123)
[19260.471840] LNet: 887186:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19261.494883] LNet: Removed LNI 10.240.22.180@tcp
[19262.518892] LNet: Removed LNI 10.240.22.180@tcp1
[19262.534983] LNet: Removed LNI 10.1.2.3@tcp99
[19262.542471] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19262.542471] lctl dl | grep ' ST ' || true
[19262.666904] Key type .llcrypt unregistered
[19262.667783] Key type ._llcrypt unregistered
[19262.824290] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[19262.922096] Key type ._llcrypt registered
[19262.926523] Key type .llcrypt registered
[19263.003738] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19263.777392] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19263.786624] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19263.790520] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19263.791838] LNet: Accept all, port 7988
[19263.884688] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[19263.888216] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[19264.175563] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim_nid 10.1.2.3@tcp --lock_prim
[19264.182505] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 4 --nid 10.1.2.3@tcp
[19264.189295] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer set --nid 10.1.2.3@tcp --state 1048577
[19264.192918] LNetError: 887817:0:(peer.c:2184:lnet_del_peer_ni()) peer 10.1.2.3@tcp created by Lustre. Must preserve primary NID, but will remove other NIDs
[19264.198526] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 4 --nid 10.1.2.3@tcp
[19264.205221] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[19264.266495] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer show -v 4 --nid 10.1.2.3@tcp
[19264.278531] LNet: 887837:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19265.302859] LNet: Removed LNI 10.240.22.180@tcp
[19265.318914] LNet: Removed LNI 10.1.2.3@tcp
[19265.326539] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19265.326539] lctl dl | grep ' ST ' || true
[19265.454904] Key type .llcrypt unregistered
[19265.455769] Key type ._llcrypt unregistered
[19267.151857] Key type ._llcrypt registered
[19267.152799] Key type .llcrypt registered
[19267.537962] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 04:52:11 \(1734411131\)
[19267.895990] Lustre: DEBUG MARKER: == sanity-lnet test 305: Resolve hostname before lnetctl ping ========================================================== 04:52:11 (1734411131)
[19267.957474] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19268.726755] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19268.735790] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19268.739561] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19268.740704] LNet: Accept all, port 7988
[19268.838739] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping 10.240.22.180@tcp
[19268.852199] Lustre: DEBUG MARKER: /usr/sbin/lnetctl ping onyx-138vm1@tcp
[19270.620658] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 310: Set timeout and verify ========== 04:52:14 \(1734411134\)
[19270.980573] Lustre: DEBUG MARKER: == sanity-lnet test 310: Set timeout and verify ========== 04:52:14 (1734411134)
[19270.998013] LNet: 888995:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19271.022932] LNet: Removed LNI 10.240.22.180@tcp
[19271.029496] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19271.029496] lctl dl | grep ' ST ' || true
[19271.162860] Key type .llcrypt unregistered
[19271.163782] Key type ._llcrypt unregistered
[19271.390751] Key type ._llcrypt registered
[19271.392303] Key type .llcrypt registered
[19271.439543] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19272.210096] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure --all
[19272.214123] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19272.215603] LNet: Accept all, port 7988
[19272.219672] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show -v
[19273.987692] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 04:52:17 \(1734411137\)
[19274.340706] Lustre: DEBUG MARKER: == sanity-lnet test 311: Fail bulk put in send wait completion ========================================================== 04:52:17 (1734411137)
[19274.743733] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_311 Need kfi network type
[19275.112759] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_311 Need kfi network type
[19275.557499] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 04:52:19 \(1734411139\)
[19275.939240] Lustre: DEBUG MARKER: == sanity-lnet test 312: TAG_RX_OK is possible after TX_FAIL ========================================================== 04:52:19 (1734411139)
[19276.338597] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_312 Need kfi network type
[19276.703389] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_312 Need kfi network type
[19277.130571] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 400: Check for udsp add\/delete net rule without net num ========================================================== 04:52:20 \(1734411140\)
[19277.486566] Lustre: DEBUG MARKER: == sanity-lnet test 400: Check for udsp add/delete net rule without net num ========================================================== 04:52:20 (1734411140)
[19277.506868] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19277.510281] LNet: 890073:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19277.518953] LNet: Removed LNI 10.240.22.180@tcp
[19277.522162] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19277.530649] Lustre: DEBUG MARKER: /usr/sbin/lnetctl udsp add --src tcp --priority 0
[19277.536049] Lustre: DEBUG MARKER: /usr/sbin/lnetctl udsp del --idx 0
[19277.544363] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19277.544363] lctl dl | grep ' ST ' || true
[19277.670857] Key type .llcrypt unregistered
[19277.671740] Key type ._llcrypt unregistered
[19279.298729] Key type ._llcrypt registered
[19279.302632] Key type .llcrypt registered
[19279.689682] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 04:52:23 \(1734411143\)
[19280.045091] Lustre: DEBUG MARKER: == sanity-lnet test 401: Discover peer after adding peer net UDSP rule ========================================================== 04:52:23 (1734411143)
[19280.110080] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19280.878504] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19280.888011] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19280.891917] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19280.893086] LNet: Accept all, port 7988
[19280.988626] Lustre: DEBUG MARKER: /usr/sbin/lnetctl udsp add --dst tcp --prio 1
[19280.998554] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[19282.789574] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 402: Destination net rule should not panic ========================================================== 04:52:26 \(1734411146\)
[19283.144930] Lustre: DEBUG MARKER: == sanity-lnet test 402: Destination net rule should not panic ========================================================== 04:52:26 (1734411146)
[19283.159524] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19283.163232] LNet: 891256:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19284.198885] LNet: Removed LNI 10.240.22.180@tcp
[19284.203190] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19284.212146] Lustre: DEBUG MARKER: /usr/sbin/lnetctl udsp add --dst kfi --priority 0
[19284.217752] Lustre: DEBUG MARKER: /usr/sbin/lnetctl peer add --prim 402@kfi
[19285.988735] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 04:52:29 \(1734411149\)
[19286.349982] Lustre: DEBUG MARKER: == sanity-lnet test 500: Check deadlock on ping target update ========================================================== 04:52:29 (1734411149)
[19286.369575] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19286.378176] LNet: 891709:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19286.382568] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19286.411545] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if eth0
[19286.415330] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19286.416495] LNet: Accept all, port 7988
[19286.513476] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --net tcp --if test1pg
[19286.517061] LNet: Added LNI 10.1.2.3@tcp [8/256/0/180]
[19286.628157] Lustre: DEBUG MARKER: /usr/sbin/lnetctl discover 10.240.22.180@tcp
[19286.660211] IPv6: ADDRCONF(NETDEV_CHANGE): test1pg: link becomes ready
[19286.671006] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19286.671006] lctl dl | grep ' ST ' || true
[19286.715314] LNetError: 891734:0:(lib-move.c:4909:lnet_parse()) 10.240.22.180@tcp, src 10.240.22.180@tcp: Dropping PUT (error -108 looking up sender)
[19287.670787] LNet: 892017:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19288.694731] LNet: Removed LNI 10.240.22.180@tcp
[19288.710802] LNet: Removed LNI 10.1.2.3@tcp
[19288.810828] Key type .llcrypt unregistered
[19288.811693] Key type ._llcrypt unregistered
[19290.467275] Key type ._llcrypt registered
[19290.468207] Key type .llcrypt registered
[19290.849425] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 501: Verify lnetctl peer set --health \(SR\) ========================================================== 04:52:34 \(1734411154\)
[19291.209804] Lustre: DEBUG MARKER: == sanity-lnet test 501: Verify lnetctl peer set --health (SR) ========================================================== 04:52:34 (1734411154)
[19291.271984] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[19292.044165] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19292.114093] LNet: 892692:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19292.121196] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19292.121196] lctl dl | grep ' ST ' || true
[19292.250790] Key type .llcrypt unregistered
[19292.251675] Key type ._llcrypt unregistered
[19292.459740] Key type ._llcrypt registered
[19292.460618] Key type .llcrypt registered
[19292.526966] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19293.302177] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[19293.306587] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19293.307760] LNet: Accept all, port 7988
[19294.033089] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_501 Need non-routed configuration
[19294.394763] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_501 Need non-routed configuration
[19294.824020] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test 502: Verify lnetctl peer set --health \(MR\) ========================================================== 04:52:38 \(1734411158\)
[19295.182330] Lustre: DEBUG MARKER: == sanity-lnet test 502: Verify lnetctl peer set --health (MR) ========================================================== 04:52:38 (1734411158)
[19295.201476] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet unconfigure
[19295.205120] LNet: 893109:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19295.214760] LNet: Removed LNI 10.240.22.180@tcp
[19295.220243] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[19295.299008] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19295.299008] lctl dl | grep ' ST ' || true
[19295.422723] Key type .llcrypt unregistered
[19295.423603] Key type ._llcrypt unregistered
[19295.602536] Key type ._llcrypt registered
[19295.603443] Key type .llcrypt registered
[19295.657629] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19296.426832] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure -a
[19296.431390] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19296.432535] LNet: Accept all, port 7988
[19297.165429] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-lnet test_502 Need non-routed configuration
[19297.526909] Lustre: DEBUG MARKER: SKIP: sanity-lnet test_502 Need non-routed configuration
[19297.953703] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-lnet test complete, duration 1085 sec ========== 04:52:41 \(1734411161\)
[19298.311018] Lustre: DEBUG MARKER: == sanity-lnet test complete, duration 1085 sec ========== 04:52:41 (1734411161)
[19298.366291] LNet: 893597:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[19298.378833] LNet: Removed LNI 10.240.22.180@tcp
[19298.387719] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[19298.387719] lctl dl | grep ' ST ' || true
[19298.566745] Key type .llcrypt unregistered
[19298.567656] Key type ._llcrypt unregistered
[19303.468556] Key type ._llcrypt registered
[19303.472463] Key type .llcrypt registered
[19303.533573] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[19304.362309] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[19304.417616] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[19304.418843] LNet: Accept all, port 7988
[19305.954619] Key type lgssc registered
[19306.126975] Lustre: Echo OBD driver; http://www.lustre.org/
[19370.055501] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[19370.066162] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[19371.107576] Lustre: Mounted lustre-client
[19371.449670] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[19372.130010] Lustre: DEBUG MARKER: 
[19372.130010] running=$(mount | grep -c /mnt/lustre' ');
[19372.130010] rc=0;
[19372.130010] if [ $running -eq 0 ] ; then
[19372.130010] 	mkdir -p /mnt/lustre;
[19372.130010] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[19372.130010] 	rc=$?;
[19372.130010] fi;
[19372.130010] exit $rc
[19374.818164] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[19379.588339] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[19379.947446] Lustre: DEBUG MARKER: Using TIMEOUT=20
[19380.355163] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[19381.047450] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[19381.415429] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[19381.426599] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[19385.223070] Autotest: Test running for 320 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[19389.369177] Lustre: DEBUG MARKER: /usr/sbin/lctl mark -----============= acceptance-small: sanity-sec ============----- Tue Dec 17 04:54:12 UTC 2024
[19389.723032] Lustre: DEBUG MARKER: -----============= acceptance-small: sanity-sec ============----- Tue Dec 17 04:54:12 UTC 2024
[19393.123123] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n version 2>/dev/null
[19395.941414] Lustre: DEBUG MARKER: cat /etc/system-release
[19395.953254] Lustre: DEBUG MARKER: test -r /etc/os-release
[19395.964171] Lustre: DEBUG MARKER: cat /etc/os-release
[19398.476790] Lustre: DEBUG MARKER: /usr/sbin/lctl mark excepting tests: 
[19398.828103] Lustre: DEBUG MARKER: excepting tests:
[19399.224918] Lustre: DEBUG MARKER: /usr/sbin/lctl mark skipping tests SLOW=no: 26
[19399.579830] Lustre: DEBUG MARKER: skipping tests SLOW=no: 26
[19400.107503] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: start setup 04:54:23 \(1734411263\) ===
[19400.459822] Lustre: DEBUG MARKER: === sanity-sec: start setup 04:54:23 (1734411263) ===
[19400.830744] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[19401.786083] Lustre: lustre-OST0000-osc-ffff988d12e52000: disconnect after 23s idle
[19402.072131] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing check_config_client \/mnt\/lustre
[19402.591821] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing check_config_client /mnt/lustre
[19406.350645] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[19406.350645] mpts=$(mount | grep -c /mnt/lustre' ');
[19406.350645] if [ $running -ne $mpts ]; then
[19406.350645]     echo $(hostname) env are INSANE!;
[19406.350645]     exit 1;
[19406.350645] fi
[19407.032569] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[19407.032569] mpts=$(mount | grep -c /mnt/lustre2' ');
[19407.032569] if [ $running -ne $mpts ]; then
[19407.032569]     echo $(hostname) env are INSANE!;
[19407.032569]     exit 1;
[19407.032569] fi
[19414.344041] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[19414.699648] Lustre: DEBUG MARKER: Using TIMEOUT=20
[19415.097044] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[19415.774888] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[19420.442162] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: finish setup 04:54:43 \(1734411283\) ===
[19420.800353] Lustre: DEBUG MARKER: === sanity-sec: finish setup 04:54:43 (1734411283) ===
[19422.504998] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 0: uid permission ======================================================================================= 04:54:46 \(1734411286\)
[19422.861447] Lustre: DEBUG MARKER: == sanity-sec test 0: uid permission ======================================================================================= 04:54:46 (1734411286)
[19424.856206] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 1: setuid\/gid ======================================================================================= 04:54:48 \(1734411288\)
[19425.214989] Lustre: DEBUG MARKER: == sanity-sec test 1: setuid/gid ======================================================================================= 04:54:48 (1734411288)
[19425.611629] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-sec test_1 without GSS support.
[19425.969989] Lustre: DEBUG MARKER: SKIP: sanity-sec test_1 without GSS support.
[19426.392756] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 4: set supplementary group ========================================================================= 04:54:49 \(1734411289\)
[19426.750297] Lustre: DEBUG MARKER: == sanity-sec test 4: set supplementary group ========================================================================= 04:54:49 (1734411289)
[19436.641724] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 7: nodemap create and delete ========== 04:55:00 \(1734411300\)
[19436.999571] Lustre: DEBUG MARKER: == sanity-sec test 7: nodemap create and delete ========== 04:55:00 (1734411300)
[19442.741833] Lustre: lustre-OST0001-osc-ffff988d12e52000: disconnect after 20s idle
[19442.743384] Lustre: Skipped 6 previous similar messages
[19447.861870] Lustre: lustre-OST0002-osc-ffff988d12e52000: disconnect after 24s idle
[19447.863500] Lustre: Skipped 1 previous similar message
[19459.427212] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 8: nodemap reject duplicates ========== 04:55:22 \(1734411322\)
[19459.783708] Lustre: DEBUG MARKER: == sanity-sec test 8: nodemap reject duplicates ========== 04:55:22 (1734411322)
[19486.861178] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 9: nodemap range add ================== 04:55:50 \(1734411350\)
[19487.219586] Lustre: DEBUG MARKER: == sanity-sec test 9: nodemap range add ================== 04:55:50 (1734411350)
[19513.546531] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10a: nodemap reject duplicate ranges == 04:56:17 \(1734411377\)
[19513.899763] Lustre: DEBUG MARKER: == sanity-sec test 10a: nodemap reject duplicate ranges == 04:56:17 (1734411377)
[19541.920135] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10b: delete range from the correct nodemap ========================================================== 04:56:45 \(1734411405\)
[19542.274120] Lustre: DEBUG MARKER: == sanity-sec test 10b: delete range from the correct nodemap ========================================================== 04:56:45 (1734411405)
[19547.149454] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10c: verfify contiguous range support ========================================================== 04:56:50 \(1734411410\)
[19547.508882] Lustre: DEBUG MARKER: == sanity-sec test 10c: verfify contiguous range support ========================================================== 04:56:50 (1734411410)
[19551.151604] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 10d: verfify nodemap range format \'\*@\<net\>\' support ========================================================== 04:56:54 \(1734411414\)
[19551.514710] Lustre: DEBUG MARKER: == sanity-sec test 10d: verfify nodemap range format '*@<net>' support ========================================================== 04:56:54 (1734411414)
[19555.158563] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 11: nodemap modify ==================== 04:56:58 \(1734411418\)
[19555.512598] Lustre: DEBUG MARKER: == sanity-sec test 11: nodemap modify ==================== 04:56:58 (1734411418)
[19581.786082] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 12: nodemap set squash ids ============ 04:57:25 \(1734411445\)
[19582.143587] Lustre: DEBUG MARKER: == sanity-sec test 12: nodemap set squash ids ============ 04:57:25 (1734411445)
[19607.765062] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 13: test nids ========================= 04:57:51 \(1734411471\)
[19608.120249] Lustre: DEBUG MARKER: == sanity-sec test 13: test nids ========================= 04:57:51 (1734411471)
[19636.413145] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 14: test default nodemap nid lookup === 04:58:19 \(1734411499\)
[19636.773200] Lustre: DEBUG MARKER: == sanity-sec test 14: test default nodemap nid lookup === 04:58:19 (1734411499)
[19662.875887] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 15: test id mapping =================== 04:58:46 \(1734411526\)
[19663.239553] Lustre: DEBUG MARKER: == sanity-sec test 15: test id mapping =================== 04:58:46 (1734411526)
[19685.914139] Autotest: Test running for 325 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[19730.157254] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 16: test nodemap all_off fileops ====== 04:59:53 \(1734411593\)
[19730.511452] Lustre: DEBUG MARKER: == sanity-sec test 16: test nodemap all_off fileops ====== 04:59:53 (1734411593)
[19730.897057] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[19743.461759] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d16.sanity-sec
[19743.473731] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d16.sanity-sec
[19743.485419] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d16.sanity-sec
[19744.110802] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[19746.499002] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19746.565570] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19747.221573] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19747.257595] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19747.909385] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19747.937722] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[19749.357666] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19749.376760] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19757.150516] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19757.808647] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19757.823716] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19758.471921] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19759.122336] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19759.144126] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[19760.482979] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19760.503038] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19772.300787] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19774.854989] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d16.sanity-sec
[19774.866148] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d16.sanity-sec
[19774.877234] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d16.sanity-sec
[19775.499979] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[19777.556439] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19777.571411] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19778.220775] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19778.235152] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19778.897495] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19778.915229] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[19780.210238] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19780.228309] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19788.011042] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19788.678696] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19788.694012] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19789.348082] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19790.003916] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19790.023593] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d16.sanity-sec/f16.sanity-sec oflag=sync bs=1M count=1
[19791.945149] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19791.964889] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d16.sanity-sec/f16.sanity-sec
[19802.415124] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19809.196090] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 17: test nodemap trusted_noadmin fileops ========================================================== 05:01:12 \(1734411672\)
[19809.550793] Lustre: DEBUG MARKER: == sanity-sec test 17: test nodemap trusted_noadmin fileops ========================================================== 05:01:12 (1734411672)
[19809.938762] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[19826.739536] Lustre: lustre-OST0001-osc-ffff988d12e52000: disconnect after 24s idle
[19833.204680] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[19833.216343] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[19833.228037] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d17.sanity-sec
[19833.871882] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[19841.224823] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19841.291037] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19841.943149] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19842.595249] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19842.616870] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19843.870870] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19843.889278] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19854.461738] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19855.205534] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19855.220348] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19855.866753] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19856.513956] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19856.533326] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19858.741064] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19858.762633] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19869.368814] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19874.788623] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[19874.801816] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[19874.813148] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d17.sanity-sec
[19875.466256] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[19880.240038] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19880.255705] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19880.913057] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19881.576302] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19881.739240] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19883.116900] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19883.136226] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19890.964106] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19891.620565] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19891.635307] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19892.280701] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19892.926108] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19892.946599] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19894.487093] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19894.506901] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19906.333336] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19913.937944] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[19913.951357] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[19913.962490] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d17.sanity-sec
[19914.593285] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[19919.302182] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19919.317296] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19919.962919] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19920.612299] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19920.631323] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19921.634624] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19921.653345] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19930.746038] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19931.484842] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19931.499754] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19932.144781] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19932.792206] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19932.811658] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19934.015577] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19934.034453] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19941.793004] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19947.149333] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d17.sanity-sec
[19947.162384] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d17.sanity-sec
[19947.173452] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d17.sanity-sec
[19947.801218] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[19952.538546] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19952.554052] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19953.199472] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19953.848975] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19953.970947] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19955.494782] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19955.514610] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19967.342242] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[19968.103838] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19968.118842] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19968.769106] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19969.420496] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19969.443347] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d17.sanity-sec/f17.sanity-sec oflag=sync bs=1M count=1
[19970.985547] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19971.004835] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d17.sanity-sec/f17.sanity-sec
[19982.846454] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[19986.959415] Autotest: Test running for 330 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[19992.480803] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 18: test nodemap mapped_noadmin fileops ========================================================== 05:04:16 \(1734411856\)
[19992.848961] Lustre: DEBUG MARKER: == sanity-sec test 18: test nodemap mapped_noadmin fileops ========================================================== 05:04:16 (1734411856)
[19993.239709] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20005.938466] Lustre: lustre-OST0001-osc-ffff988d12e52000: disconnect after 23s idle
[20005.940098] Lustre: Skipped 4 previous similar messages
[20019.522431] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d18.sanity-sec
[20019.535600] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d18.sanity-sec
[20019.546966] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d18.sanity-sec
[20020.232701] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20027.579455] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20027.678017] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20028.324190] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20028.974547] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20028.994523] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[20030.841109] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20030.859129] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20038.666965] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20039.325271] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20039.340504] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20040.006135] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20040.665496] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20040.686892] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[20042.197084] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20042.216097] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20048.649659] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20053.928972] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d18.sanity-sec
[20053.942539] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d18.sanity-sec
[20053.953710] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d18.sanity-sec
[20054.581281] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20059.323070] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20059.338704] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20059.993350] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20060.654566] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20060.673114] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[20062.792185] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20062.810046] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20074.628202] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20075.372898] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20076.031061] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20076.046553] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20076.704201] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20076.723896] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d18.sanity-sec/f18.sanity-sec oflag=sync bs=1M count=1
[20078.532949] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20078.553200] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d18.sanity-sec/f18.sanity-sec
[20090.525141] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20100.136807] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 19: test nodemap trusted_admin fileops ========================================================== 05:06:03 \(1734411963\)
[20100.493970] Lustre: DEBUG MARKER: == sanity-sec test 19: test nodemap trusted_admin fileops ========================================================== 05:06:03 (1734411963)
[20100.874235] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20113.457833] Lustre: lustre-OST0000-osc-ffff988d12e52000: disconnect after 23s idle
[20113.459427] Lustre: Skipped 4 previous similar messages
[20124.189239] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d19.sanity-sec
[20124.201157] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d19.sanity-sec
[20124.212532] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d19.sanity-sec
[20124.856461] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20132.302558] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20132.394537] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20133.050658] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20133.723412] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20134.383371] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20134.403031] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[20135.987947] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20136.005899] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20146.485277] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20147.230780] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20147.246273] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20147.898388] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20148.556111] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20148.575839] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[20149.975414] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20149.994586] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20161.895541] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20167.330359] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d19.sanity-sec
[20167.341701] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d19.sanity-sec
[20167.355534] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d19.sanity-sec
[20167.991371] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20172.856583] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20172.872109] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20173.517903] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20173.533210] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20174.183793] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20174.202139] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[20176.112810] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20176.131316] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20187.977332] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20188.722683] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20188.738116] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20189.392750] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20190.046705] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20190.066847] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d19.sanity-sec/f19.sanity-sec oflag=sync bs=1M count=1
[20191.394590] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20191.414954] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d19.sanity-sec/f19.sanity-sec
[20203.371173] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20213.069447] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 20: test nodemap mapped_admin fileops ========================================================== 05:07:56 \(1734412076\)
[20213.426277] Lustre: DEBUG MARKER: == sanity-sec test 20: test nodemap mapped_admin fileops ========================================================== 05:07:56 (1734412076)
[20213.805217] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20226.097169] Lustre: lustre-OST0000-osc-ffff988d12e52000: disconnect after 23s idle
[20226.098742] Lustre: Skipped 6 previous similar messages
[20237.299406] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d20.sanity-sec
[20237.311780] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d20.sanity-sec
[20237.323496] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d20.sanity-sec
[20237.981239] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20245.330319] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20245.375706] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20246.023120] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20246.115112] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20246.773972] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20246.793657] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[20247.897406] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20247.915677] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20259.708543] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20260.452470] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20260.467474] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20261.125610] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20261.784243] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20261.803724] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[20263.013486] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20263.037369] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20274.895498] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20280.287017] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d20.sanity-sec
[20280.298245] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d20.sanity-sec
[20280.309151] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d20.sanity-sec
[20280.945925] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20285.738719] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20285.754308] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20286.406240] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20286.424104] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20287.077640] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20287.095974] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[20288.133718] Autotest: Test running for 335 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[20288.199584] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20288.217989] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20300.050992] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20300.792733] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20301.445312] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20301.460329] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20302.119806] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20302.138714] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d20.sanity-sec/f20.sanity-sec oflag=sync bs=1M count=1
[20304.133424] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20304.152814] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d20.sanity-sec/f20.sanity-sec
[20315.987546] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20325.432188] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 21: test nodemap mapped_trusted_noadmin fileops ========================================================== 05:09:48 \(1734412188\)
[20325.784926] Lustre: DEBUG MARKER: == sanity-sec test 21: test nodemap mapped_trusted_noadmin fileops ========================================================== 05:09:48 (1734412188)
[20326.159560] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20338.736496] Lustre: lustre-OST0000-osc-ffff988d12e52000: disconnect after 23s idle
[20338.738055] Lustre: Skipped 6 previous similar messages
[20347.216869] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d21.sanity-sec
[20347.231333] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d21.sanity-sec
[20347.242338] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d21.sanity-sec
[20347.879303] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20355.306084] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20356.000578] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20356.660628] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20357.314013] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20357.334132] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[20358.502451] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20358.520397] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20366.350397] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20367.012143] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20367.027454] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20367.680188] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20368.336816] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20368.356791] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[20371.234178] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20371.253915] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20381.809946] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20387.210509] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d21.sanity-sec
[20387.223743] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d21.sanity-sec
[20387.234868] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d21.sanity-sec
[20387.871934] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20392.624773] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20392.640452] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20393.289547] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20393.942247] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20393.960661] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[20395.984796] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20396.002859] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20407.800584] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20408.543275] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20408.557973] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20409.203789] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20409.851820] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20409.870966] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d21.sanity-sec/f21.sanity-sec oflag=sync bs=1M count=1
[20412.293875] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20412.316200] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d21.sanity-sec/f21.sanity-sec
[20422.818269] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20432.434636] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 22: test nodemap mapped_trusted_admin fileops ========================================================== 05:11:35 \(1734412295\)
[20432.798021] Lustre: DEBUG MARKER: == sanity-sec test 22: test nodemap mapped_trusted_admin fileops ========================================================== 05:11:35 (1734412295)
[20433.175658] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20446.255888] Lustre: lustre-OST0000-osc-ffff988d12e52000: disconnect after 24s idle
[20446.257441] Lustre: Skipped 6 previous similar messages
[20454.737919] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d22.sanity-sec
[20454.749550] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d22.sanity-sec
[20454.760779] Lustre: DEBUG MARKER: chown 0 /mnt/lustre/d22.sanity-sec
[20455.398877] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20460.504958] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20461.565685] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20462.215171] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20462.262280] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20462.923007] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20462.943254] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[20464.046170] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20464.064826] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20474.541524] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20475.296939] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20475.312151] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20475.967843] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20476.621355] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20476.641684] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[20477.773670] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20477.793981] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20489.615687] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20494.907577] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d22.sanity-sec
[20494.918955] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d22.sanity-sec
[20494.929838] Lustre: DEBUG MARKER: chown 60000 /mnt/lustre/d22.sanity-sec
[20495.571928] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20500.301739] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20500.316992] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20500.969645] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20500.984060] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20501.634800] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20501.652327] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[20503.229283] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20503.246940] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20515.060450] Lustre: DEBUG MARKER: runas -u0 -g0 -G0 lfs quota -q /mnt/lustre
[20515.795551] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20515.810620] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20516.459438] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 touch /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20517.105164] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20517.124840] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 dd if=/dev/zero of=/mnt/lustre/d22.sanity-sec/f22.sanity-sec oflag=sync bs=1M count=1
[20519.153964] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20519.173148] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 rm /mnt/lustre/d22.sanity-sec/f22.sanity-sec
[20531.049918] Lustre: DEBUG MARKER: runas -u60003 -g60003 -G60003 lfs quota -q /mnt/lustre
[20540.656395] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 23a: test mapped regular ACLs ========= 05:13:24 \(1734412404\)
[20541.011554] Lustre: DEBUG MARKER: == sanity-sec test 23a: test mapped regular ACLs ========= 05:13:24 (1734412404)
[20541.397702] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20562.700904] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[20562.713027] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[20562.726660] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[20570.343489] Lustre: DEBUG MARKER: runas -u 60000 touch /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20570.534367] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20571.361596] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20571.793908] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20571.813165] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20572.139368] bash (977462): drop_caches: 3
[20577.203975] Lustre: DEBUG MARKER: runas -u 60000 setfacl -m 60000:rwx /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20577.263932] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20578.102123] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20578.453665] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20578.472714] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20578.722230] bash (977740): drop_caches: 3
[20587.923546] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[20587.956388] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[20587.968891] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[20589.298078] Autotest: Test running for 340 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[20595.634014] Lustre: DEBUG MARKER: runas -u 60002 touch /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20596.185105] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20597.027675] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20597.380893] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20597.400823] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20597.671267] bash (978711): drop_caches: 3
[20599.855014] Lustre: lustre-OST0005-osc-ffff988d12e52000: disconnect after 21s idle
[20599.856595] Lustre: Skipped 13 previous similar messages
[20601.555541] Lustre: DEBUG MARKER: runas -u 60002 setfacl -m 60002:rwx /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20601.574000] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20602.425205] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20602.781898] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20602.801526] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20603.108009] bash (978988): drop_caches: 3
[20612.984308] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[20613.000757] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[20613.012598] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[20626.007510] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20626.024345] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20626.871475] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20627.221208] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20627.240877] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20627.477472] bash (979951): drop_caches: 3
[20632.577300] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20632.719059] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20633.563934] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20633.917414] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20633.937032] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20634.208558] bash (980228): drop_caches: 3
[20639.528108] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[20639.611762] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[20639.623616] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[20651.284078] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20651.299860] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20652.143626] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20652.497062] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20652.516573] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20652.916937] bash (981132): drop_caches: 3
[20665.026897] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[20665.045048] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[20665.056722] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[20683.053249] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20683.072850] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20683.922821] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20684.280924] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20684.299758] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20685.195464] bash (982294): drop_caches: 3
[20696.758354] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20696.775217] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20697.620149] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20698.220663] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20698.240951] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20698.642331] bash (982571): drop_caches: 3
[20703.880272] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23a.sanity-sec
[20704.150618] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23a.sanity-sec
[20704.162423] Lustre: DEBUG MARKER: chmod a+rwx /mnt/lustre/d23a.sanity-sec
[20719.118049] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20719.134997] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20719.983692] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20720.337680] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20720.358090] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20720.651405] bash (983475): drop_caches: 3
[20724.471747] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23a.sanity-sec/f23a.sanity-sec
[20724.488168] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
[20725.339868] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20725.696633] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing cancel_lru_locks
[20725.719369] Lustre: DEBUG MARKER: sync ; echo 3 > /proc/sys/vm/drop_caches
[20726.053290] bash (983753): drop_caches: 3
[20735.569804] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 23b: test mapped default ACLs ========= 05:16:39 \(1734412599\)
[20735.931213] Lustre: DEBUG MARKER: == sanity-sec test 23b: test mapped default ACLs ========= 05:16:39 (1734412599)
[20736.322064] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20760.845730] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/d23b.sanity-sec
[20760.859122] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre/d23b.sanity-sec
[20760.943853] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23b.sanity-sec
[20760.956126] Lustre: DEBUG MARKER: setfacl -R -d -m group:sanityusr:rwx /mnt/lustre/d23b.sanity-sec
[20760.970281] Lustre: DEBUG MARKER: sync && stat /mnt/lustre/d23b.sanity-sec > /dev/null
[20761.765426] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param -t4 -n ldlm.namespaces.*.lru_size=clear
[20761.777985] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23b.sanity-sec
[20761.792112] Lustre: DEBUG MARKER: getfacl /mnt/lustre/d23b.sanity-sec
[20775.016746] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 24: check nodemap proc files for LBUGs and Oopses ========================================================== 05:17:18 \(1734412638\)
[20775.377796] Lustre: DEBUG MARKER: == sanity-sec test 24: check nodemap proc files for LBUGs and Oopses ========================================================== 05:17:18 (1734412638)
[20775.748014] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20800.293318] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 25: test save and reload nodemap config ========================================================== 05:17:43 \(1734412663\)
[20800.649769] Lustre: DEBUG MARKER: == sanity-sec test 25: test save and reload nodemap config ========================================================== 05:17:43 (1734412663)
[20800.998658] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[20800.998658] if [ $running -ne 0 ] ; then
[20800.998658] echo Stopping client $(hostname) /mnt/lustre opts:;
[20800.998658] lsof /mnt/lustre || need_kill=no;
[20800.998658] if [ x != x -a x$need_kill != xno ]; then
[20800.998658]     pids=$(lsof -t /mnt/lustre | sort -u);
[20800.998658]     if 
[20802.130563] Lustre: Unmounted lustre-client
[20802.531390] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[20822.194335] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: start setup 05:18:05 \(1734412685\) ===
[20822.629273] Lustre: DEBUG MARKER: === sanity-sec: start setup 05:18:05 (1734412685) ===
[20826.256161] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[20826.256161] mpts=$(mount | grep -c /mnt/lustre' ');
[20826.256161] if [ $running -ne $mpts ]; then
[20826.256161]     echo $(hostname) env are INSANE!;
[20826.256161]     exit 1;
[20826.256161] fi
[20826.939332] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[20826.939332] mpts=$(mount | grep -c /mnt/lustre2' ');
[20826.939332] if [ $running -ne $mpts ]; then
[20826.939332]     echo $(hostname) env are INSANE!;
[20826.939332]     exit 1;
[20826.939332] fi
[20853.605028] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[20853.615707] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[20853.665297] Lustre: Mounted lustre-client
[20854.004877] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[20854.816870] Lustre: DEBUG MARKER: 
[20854.816870] running=$(mount | grep -c /mnt/lustre' ');
[20854.816870] rc=0;
[20854.816870] if [ $running -eq 0 ] ; then
[20854.816870] 	mkdir -p /mnt/lustre;
[20854.816870] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[20854.816870] 	rc=$?;
[20854.816870] fi;
[20854.816870] exit $rc
[20855.542901] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[20860.367390] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20
[20860.724910] Lustre: DEBUG MARKER: Using TIMEOUT=20
[20861.123459] Lustre: DEBUG MARKER: [ -f /sys/module/mgc/parameters/mgc_requeue_timeout_min ] && echo 1 > /sys/module/mgc/parameters/mgc_requeue_timeout_min; exit 0
[20861.807963] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l
[20862.180948] Lustre: DEBUG MARKER: sysctl --values kernel/kptr_restrict
[20862.191992] Lustre: DEBUG MARKER: sysctl -wq kernel/kptr_restrict=1
[20871.282198] Lustre: DEBUG MARKER: /usr/sbin/lctl mark === sanity-sec: finish setup 05:18:54 \(1734412734\) ===
[20871.640455] Lustre: DEBUG MARKER: === sanity-sec: finish setup 05:18:54 (1734412734) ===
[20871.970036] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[20871.970036] if [ $running -ne 0 ] ; then
[20871.970036] echo Stopping client $(hostname) /mnt/lustre opts:;
[20871.970036] lsof /mnt/lustre || need_kill=no;
[20871.970036] if [ x != x -a x$need_kill != xno ]; then
[20871.970036]     pids=$(lsof -t /mnt/lustre | sort -u);
[20871.970036]     if 
[20873.111395] Lustre: Unmounted lustre-client
[20882.004016] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[20882.683889] Lustre: DEBUG MARKER: 
[20882.683889] running=$(mount | grep -c /mnt/lustre' ');
[20882.683889] rc=0;
[20882.683889] if [ $running -eq 0 ] ; then
[20882.683889] 	mkdir -p /mnt/lustre;
[20882.683889] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[20882.683889] 	rc=$?;
[20882.683889] fi;
[20882.683889] exit $rc
[20883.080139] Lustre: Mounted lustre-client
[20883.418076] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[20887.501822] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-sec test_26 skipping SLOW test 26
[20887.857997] Lustre: DEBUG MARKER: SKIP: sanity-sec test_26 skipping SLOW test 26
[20888.280166] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 27a: test fileset in various nodemaps ========================================================== 05:19:11 \(1734412751\)
[20888.637826] Lustre: DEBUG MARKER: == sanity-sec test 27a: test fileset in various nodemaps ========================================================== 05:19:11 (1734412751)
[20890.180971] Autotest: Test running for 345 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[20899.109985] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[20899.126932] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[20899.143211] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default/subsubdir_default
[20899.155344] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default/subsubdir_default/this_is_subsubdir_default
[20899.786101] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[20906.727601] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[20906.727601] if [ $running -ne 0 ] ; then
[20906.727601] echo Stopping client $(hostname) /mnt/lustre opts:;
[20906.727601] lsof /mnt/lustre || need_kill=no;
[20906.727601] if [ x != x -a x$need_kill != xno ]; then
[20906.727601]     pids=$(lsof -t /mnt/lustre | sort -u);
[20906.727601]     if 
[20907.937756] Lustre: Unmounted lustre-client
[20908.266731] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[20908.266731] if [ $running -ne 0 ] ; then
[20908.266731] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[20908.266731] lsof /mnt/lustre2 || need_kill=no;
[20908.266731] if [ x != x -a x$need_kill != xno ]; then
[20908.266731]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[20908.266731]    
[20942.445770] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[20942.445770] lctl dl | grep ' ST ' || true
[20942.842441] Key type lgssc unregistered
[20942.909059] LNet: 997874:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[20942.925039] LNet: Removed LNI 10.240.22.180@tcp
[20943.141544] Key type .llcrypt unregistered
[20943.142459] Key type ._llcrypt unregistered
[20949.247448] Key type ._llcrypt registered
[20949.248374] Key type .llcrypt registered
[20949.317453] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[20950.147258] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[20950.207662] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[20950.208869] LNet: Accept all, port 7988
[20951.744838] Key type lgssc registered
[20951.924321] Lustre: Echo OBD driver; http://www.lustre.org/
[21012.180267] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21012.190959] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21012.190959] 					umount /mnt/lustre
[21012.208723] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[21013.249558] Lustre: Mounted lustre-client
[21013.257726] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[21013.270111] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//
[21013.281250] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21013.281250] 			umount /mnt/lustre
[21013.325601] Lustre: Unmounted lustre-client
[21013.332671] Lustre: DEBUG MARKER: 
[21013.332671] running=$(mount | grep -c /mnt/lustre' ');
[21013.332671] rc=0;
[21013.332671] if [ $running -eq 0 ] ; then
[21013.332671] 	mkdir -p /mnt/lustre;
[21013.332671] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre/ /mnt/lustre;
[21013.332671] 	rc=$?;
[21013.332671] fi;
[21013.332671] exit $rc
[21013.387267] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21013.416757] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[21013.430182] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21013.430182] if [ $running -ne 0 ] ; then
[21013.430182] echo Stopping client $(hostname) /mnt/lustre opts:;
[21013.430182] lsof /mnt/lustre || need_kill=no;
[21013.430182] if [ x != x -a x$need_kill != xno ]; then
[21013.430182]     pids=$(lsof -t /mnt/lustre | sort -u);
[21013.430182]     if 
[21013.658762] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21013.669384] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21013.669384] 					umount /mnt/lustre
[21013.680951] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[21013.730076] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[21013.742804] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//subsubdir_default
[21013.755421] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21013.755421] 			umount /mnt/lustre
[21013.793331] Lustre: DEBUG MARKER: 
[21013.793331] running=$(mount | grep -c /mnt/lustre' ');
[21013.793331] rc=0;
[21013.793331] if [ $running -eq 0 ] ; then
[21013.793331] 	mkdir -p /mnt/lustre;
[21013.793331] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre/subsubdir_default /mnt/lustre;
[21013.793331] 	rc=$?;
[21013.793331] fi;
[21013.793331] exit $rc
[21013.843650] Lustre: Mounted lustre-client
[21013.844509] Lustre: Skipped 2 previous similar messages
[21013.849951] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21013.878718] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_subsubdir_default
[21023.840587] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21023.840587] if [ $running -ne 0 ] ; then
[21023.840587] echo Stopping client $(hostname) /mnt/lustre opts:;
[21023.840587] lsof /mnt/lustre || need_kill=no;
[21023.840587] if [ x != x -a x$need_kill != xno ]; then
[21023.840587]     pids=$(lsof -t /mnt/lustre | sort -u);
[21023.840587]     if 
[21024.049868] Lustre: Unmounted lustre-client
[21024.050813] Lustre: Skipped 2 previous similar messages
[21024.057462] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21024.067900] Lustre: DEBUG MARKER: 
[21024.067900] running=$(mount | grep -c /mnt/lustre' ');
[21024.067900] rc=0;
[21024.067900] if [ $running -eq 0 ] ; then
[21024.067900] 	mkdir -p /mnt/lustre;
[21024.067900] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[21024.067900] 	rc=$?;
[21024.067900] fi;
[21024.067900] exit $rc
[21024.120637] Lustre: Mounted lustre-client
[21024.127233] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21024.157286] Lustre: DEBUG MARKER: test -d /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[21029.410342] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_default
[21030.050769] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[21049.203731] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21049.218237] Lustre: DEBUG MARKER: 
[21049.218237] running=$(mount | grep -c /mnt/lustre' ');
[21049.218237] rc=0;
[21049.218237] if [ $running -eq 0 ] ; then
[21049.218237] 	mkdir -p /mnt/lustre;
[21049.218237] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[21049.218237] 	rc=$?;
[21049.218237] fi;
[21049.218237] exit $rc
[21049.235695] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21059.795392] Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep -w tcp | cut -f 1 -d @
[21106.182871] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[21106.197245] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[21106.252886] Lustre: DEBUG MARKER: mkdir /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0/subsubdir_c0
[21106.265826] Lustre: DEBUG MARKER: touch /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0/subsubdir_c0/this_is_subsubdir_c0
[21106.930957] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[21154.955884] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21154.955884] if [ $running -ne 0 ] ; then
[21154.955884] echo Stopping client $(hostname) /mnt/lustre opts:;
[21154.955884] lsof /mnt/lustre || need_kill=no;
[21154.955884] if [ x != x -a x$need_kill != xno ]; then
[21154.955884]     pids=$(lsof -t /mnt/lustre | sort -u);
[21154.955884]     if 
[21156.173400] Lustre: Unmounted lustre-client
[21156.514128] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[21156.514128] if [ $running -ne 0 ] ; then
[21156.514128] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[21156.514128] lsof /mnt/lustre2 || need_kill=no;
[21156.514128] if [ x != x -a x$need_kill != xno ]; then
[21156.514128]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[21156.514128]    
[21185.645105] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21185.645105] lctl dl | grep ' ST ' || true
[21186.035975] Key type lgssc unregistered
[21186.103507] LNet: 1009628:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21187.131568] LNet: Removed LNI 10.240.22.180@tcp
[21187.363684] Key type .llcrypt unregistered
[21187.364653] Key type ._llcrypt unregistered
[21192.302957] Autotest: Test running for 350 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[21192.526529] Key type ._llcrypt registered
[21192.527585] Key type .llcrypt registered
[21192.583664] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21193.399720] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[21193.450429] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[21193.451693] LNet: Accept all, port 7988
[21194.991417] Key type lgssc registered
[21195.162086] Lustre: Echo OBD driver; http://www.lustre.org/
[21255.468795] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21255.479727] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21255.479727] 					umount /mnt/lustre
[21255.491471] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[21256.532459] Lustre: Mounted lustre-client
[21256.539951] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[21256.552540] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//
[21256.563759] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21256.563759] 			umount /mnt/lustre
[21256.607179] Lustre: Unmounted lustre-client
[21256.614495] Lustre: DEBUG MARKER: 
[21256.614495] running=$(mount | grep -c /mnt/lustre' ');
[21256.614495] rc=0;
[21256.614495] if [ $running -eq 0 ] ; then
[21256.614495] 	mkdir -p /mnt/lustre;
[21256.614495] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre/ /mnt/lustre;
[21256.614495] 	rc=$?;
[21256.614495] fi;
[21256.614495] exit $rc
[21256.671659] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21256.702691] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[21256.715377] Lustre: DEBUG MARKER: /usr/bin/lfs setquota -p 1 -b 10000 -B 11000 -i 0 -I 0 /mnt/lustre
[21256.729059] Lustre: DEBUG MARKER: /usr/bin/lfs setquota -p 2 -b 10000 -B 11000 -i 0 -I 0 /mnt/lustre
[21256.737969] LustreError: 1012199:0:(mdc_request.c:2134:mdc_quotactl()) lustre-MDT0000-mdc-ffff988c12222000: ptlrpc_queue_wait failed: rc = -1
[21256.744107] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21256.744107] if [ $running -ne 0 ] ; then
[21256.744107] echo Stopping client $(hostname) /mnt/lustre opts:;
[21256.744107] lsof /mnt/lustre || need_kill=no;
[21256.744107] if [ x != x -a x$need_kill != xno ]; then
[21256.744107]     pids=$(lsof -t /mnt/lustre | sort -u);
[21256.744107]     if 
[21256.972429] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21256.982943] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21256.982943] 					umount /mnt/lustre
[21256.994425] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[21257.036918] Lustre: Mounted lustre-client
[21257.037759] Lustre: Skipped 1 previous similar message
[21257.044457] Lustre: DEBUG MARKER: lctl get_param -n mdc.lustre-MDT0000*.import
[21257.056962] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre//subsubdir_c0
[21257.069710] Lustre: DEBUG MARKER: ! grep -q /mnt/lustre' ' /proc/mounts ||
[21257.069710] 			umount /mnt/lustre
[21257.107935] Lustre: DEBUG MARKER: 
[21257.107935] running=$(mount | grep -c /mnt/lustre' ');
[21257.107935] rc=0;
[21257.107935] if [ $running -eq 0 ] ; then
[21257.107935] 	mkdir -p /mnt/lustre;
[21257.107935] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre/subsubdir_c0 /mnt/lustre;
[21257.107935] 	rc=$?;
[21257.107935] fi;
[21257.107935] exit $rc
[21257.162671] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21257.191948] Lustre: DEBUG MARKER: test -f /mnt/lustre/this_is_subsubdir_c0
[21267.179079] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21267.179079] if [ $running -ne 0 ] ; then
[21267.179079] echo Stopping client $(hostname) /mnt/lustre opts:;
[21267.179079] lsof /mnt/lustre || need_kill=no;
[21267.179079] if [ x != x -a x$need_kill != xno ]; then
[21267.179079]     pids=$(lsof -t /mnt/lustre | sort -u);
[21267.179079]     if 
[21267.393506] Lustre: Unmounted lustre-client
[21267.394445] Lustre: Skipped 2 previous similar messages
[21267.402184] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21267.414178] Lustre: DEBUG MARKER: 
[21267.414178] running=$(mount | grep -c /mnt/lustre' ');
[21267.414178] rc=0;
[21267.414178] if [ $running -eq 0 ] ; then
[21267.414178] 	mkdir -p /mnt/lustre;
[21267.414178] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[21267.414178] 	rc=$?;
[21267.414178] fi;
[21267.414178] exit $rc
[21267.463316] Lustre: Mounted lustre-client
[21267.464169] Lustre: Skipped 1 previous similar message
[21267.472670] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21267.505064] Lustre: DEBUG MARKER: test -d /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[21274.810280] Lustre: DEBUG MARKER: rm -rf /mnt/lustre/thisisaverylongsubdirtotestlongfilesetsandtotestmultiplefilesetfragmentsonthenodemapiam_c0
[21275.456198] Lustre: DEBUG MARKER: /usr/sbin/lctl set_param ldlm.namespaces.lustre-MDT*.lru_size=clear
[21307.447499] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21307.458498] Lustre: DEBUG MARKER: 
[21307.458498] running=$(mount | grep -c /mnt/lustre' ');
[21307.458498] rc=0;
[21307.458498] if [ $running -eq 0 ] ; then
[21307.458498] 	mkdir -p /mnt/lustre;
[21307.458498] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[21307.458498] 	rc=$?;
[21307.458498] fi;
[21307.458498] exit $rc
[21307.475302] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21314.370551] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 27aa: test nodemap idmap range ======== 05:26:17 \(1734413177\)
[21314.727428] Lustre: DEBUG MARKER: == sanity-sec test 27aa: test nodemap idmap range ======== 05:26:17 (1734413177)
[21319.639261] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 27b: The new nodemap won\'t clear the old nodemap\'s fileset ========================================================== 05:26:23 \(1734413183\)
[21319.996629] Lustre: DEBUG MARKER: == sanity-sec test 27b: The new nodemap won't clear the old nodemap's fileset ========================================================== 05:26:23 (1734413183)
[21320.403617] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-sec test_27b needs \>= 2 MDTs
[21320.759767] Lustre: DEBUG MARKER: SKIP: sanity-sec test_27b needs >= 2 MDTs
[21321.185523] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 28: check shared key rotation method == 05:26:24 \(1734413184\)
[21321.539557] Lustre: DEBUG MARKER: == sanity-sec test 28: check shared key rotation method == 05:26:24 (1734413184)
[21321.940491] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-sec test_28 need shared key feature for this test
[21322.295796] Lustre: DEBUG MARKER: SKIP: sanity-sec test_28 need shared key feature for this test
[21322.727446] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 29: check for missing shared key ====== 05:26:26 \(1734413186\)
[21323.085434] Lustre: DEBUG MARKER: == sanity-sec test 29: check for missing shared key ====== 05:26:26 (1734413186)
[21323.481667] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-sec test_29 need shared key feature for this test
[21323.839708] Lustre: DEBUG MARKER: SKIP: sanity-sec test_29 need shared key feature for this test
[21324.273958] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 30: check for invalid shared key ====== 05:26:27 \(1734413187\)
[21324.629798] Lustre: DEBUG MARKER: == sanity-sec test 30: check for invalid shared key ====== 05:26:27 (1734413187)
[21325.030892] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-sec test_30 need shared key feature for this test
[21325.388535] Lustre: DEBUG MARKER: SKIP: sanity-sec test_30 need shared key feature for this test
[21325.814217] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 30b: basic test of all different SSK flavors ========================================================== 05:26:29 \(1734413189\)
[21326.171994] Lustre: DEBUG MARKER: == sanity-sec test 30b: basic test of all different SSK flavors ========================================================== 05:26:29 (1734413189)
[21326.572407] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  SKIP: sanity-sec test_30b need shared key feature for this test
[21326.926097] Lustre: DEBUG MARKER: SKIP: sanity-sec test_30b need shared key feature for this test
[21327.354784] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-sec test 31: client mount option \'-o network\' == 05:26:30 \(1734413190\)
[21327.711470] Lustre: DEBUG MARKER: == sanity-sec test 31: client mount option '-o network' == 05:26:30 (1734413190)
[21328.219637] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net show --net tcp
[21329.891636] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[21329.902948] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[21330.078128] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[21330.114288] Lustre: Unmounted lustre-client
[21333.033114] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21333.033114] if [ $running -ne 0 ] ; then
[21333.033114] echo Stopping client $(hostname) /mnt/lustre opts:;
[21333.033114] lsof /mnt/lustre || need_kill=no;
[21333.033114] if [ x != x -a x$need_kill != xno ]; then
[21333.033114]     pids=$(lsof -t /mnt/lustre | sort -u);
[21333.033114]     if 
[21334.492974] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[21334.492974] if [ $running -ne 0 ] ; then
[21334.492974] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[21334.492974] lsof /mnt/lustre2 || need_kill=no;
[21334.492974] if [ x != x -a x$need_kill != xno ]; then
[21334.492974]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[21334.492974]    
[21368.712725] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21368.712725] lctl dl | grep ' ST ' || true
[21369.114712] Key type lgssc unregistered
[21369.182423] LNet: 1019100:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21369.202479] LNet: Removed LNI 10.240.22.180@tcp
[21369.426695] Key type .llcrypt unregistered
[21369.427641] Key type ._llcrypt unregistered
[21376.035016] Key type ._llcrypt registered
[21376.038619] Key type .llcrypt registered
[21376.415987] Lustre: DEBUG MARKER: /usr/sbin/lctl mark onyx-138vm1.onyx.whamcloud.com: executing load_modules
[21376.991670] Lustre: DEBUG MARKER: onyx-138vm1.onyx.whamcloud.com: executing load_modules
[21377.644535] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 1
[21378.438842] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[21378.473393] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[21378.474629] LNet: Accept all, port 7988
[21380.002277] Key type lgssc registered
[21380.135847] Lustre: Echo OBD driver; http://www.lustre.org/
[21380.678110] Lustre: DEBUG MARKER: /usr/sbin/lnetctl lnet configure
[21380.694536] Lustre: DEBUG MARKER: /usr/sbin/lnetctl net add --if eth0 --net tcp999
[21380.702809] LNet: Added LNI 10.240.22.180@tcp999 [8/256/0/180]
[21404.334665] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21404.334665] mpts=$(mount | grep -c /mnt/lustre' ');
[21404.334665] if [ $running -ne $mpts ]; then
[21404.334665]     echo $(hostname) env are INSANE!;
[21404.334665]     exit 1;
[21404.334665] fi
[21405.100081] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[21405.100081] mpts=$(mount | grep -c /mnt/lustre2' ');
[21405.100081] if [ $running -ne $mpts ]; then
[21405.100081]     echo $(hostname) env are INSANE!;
[21405.100081]     exit 1;
[21405.100081] fi
[21464.701878] Key type lgssc unregistered
[21464.798814] LNet: 1023297:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21464.800571] LNet: Removed LNI 10.240.22.180@tcp
[21464.813863] LNet: Removed LNI 10.240.22.180@tcp999
[21465.009819] Key type .llcrypt unregistered
[21465.013477] Key type ._llcrypt unregistered
[21465.107380] Key type ._llcrypt registered
[21465.108243] Key type .llcrypt registered
[21465.183114] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21465.995567] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[21466.032231] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[21466.033462] LNet: Accept all, port 7988
[21467.565749] Key type lgssc registered
[21467.718659] Lustre: Echo OBD driver; http://www.lustre.org/
[21467.768680] LNet: Added LNI 10.240.22.180@tcp999 [8/256/0/180]
[21467.877402] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21467.887689] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,network=tcp999 10.240.22.189@tcp999:/lustre /mnt/lustre
[21473.001741] LustreError: 1023683:0:(mgc_request.c:232:do_config_log_add()) MGC10.240.22.189@tcp999: failed processing log, type 1: rc = -5
[21477.156030] Lustre: Mounted lustre-client
[21478.269568] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[21478.280925] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[21478.479467] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[21478.513487] Lustre: Unmounted lustre-client
[21480.933949] Key type lgssc unregistered
[21481.026681] LNet: 1023991:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21481.028431] LNet: Removed LNI 10.240.22.180@tcp
[21481.045817] LNet: Removed LNI 10.240.22.180@tcp999
[21481.249776] Key type .llcrypt unregistered
[21481.250726] Key type ._llcrypt unregistered
[21481.382835] Key type ._llcrypt registered
[21481.383701] Key type .llcrypt registered
[21481.464160] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21482.272662] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[21482.310481] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[21482.311691] LNet: Accept all, port 7988
[21483.841650] Key type lgssc registered
[21483.992857] Lustre: Echo OBD driver; http://www.lustre.org/
[21484.050199] LNet: Added LNI 10.240.22.180@tcp999 [8/256/0/180]
[21484.158505] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21484.168818] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,network=tcp999 10.240.22.189@tcp999:/lustre /mnt/lustre
[21484.179312] LustreError: 1024378:0:(obd_mount.c:1723:lmd_parse()) LNet Dynamic Peer Discovery is enabled on this node. 'network' mount option cannot be taken into account.
[21484.182142] LustreError: 1024378:0:(obd_mount.c:1796:lmd_parse()) Bad mount options user_xattr,flock,
[21484.183837] LustreError: 1024378:0:(super25.c:188:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -22
[21485.661756] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[21485.990973] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21485.990973] if [ $running -ne 0 ] ; then
[21485.990973] echo Stopping client $(hostname) /mnt/lustre opts:;
[21485.990973] lsof /mnt/lustre || need_kill=no;
[21485.990973] if [ x != x -a x$need_kill != xno ]; then
[21485.990973]     pids=$(lsof -t /mnt/lustre | sort -u);
[21485.990973]     if 
[21486.674115] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[21486.674115] if [ $running -ne 0 ] ; then
[21486.674115] echo Stopping client $(hostname) /mnt/lustre2 opts:;
[21486.674115] lsof /mnt/lustre2 || need_kill=no;
[21486.674115] if [ x != x -a x$need_kill != xno ]; then
[21486.674115]     pids=$(lsof -t /mnt/lustre2 | sort -u);
[21486.674115]    
[21493.619154] Autotest: Test running for 355 minutes (lustre-reviews_review-ldiskfs-ubuntu_109661.43)
[21517.742167] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null &&
[21517.742167] lctl dl | grep ' ST ' || true
[21518.134276] Key type lgssc unregistered
[21518.222429] LNet: 1025448:0:(lib-ptl.c:966:lnet_clear_lazy_portal()) Active lazy portal 0 on exit
[21518.224190] LNet: Removed LNI 10.240.22.180@tcp
[21518.237591] LNet: Removed LNI 10.240.22.180@tcp999
[21518.445498] Key type .llcrypt unregistered
[21518.446401] Key type ._llcrypt unregistered
[21524.596321] Key type ._llcrypt registered
[21524.597288] Key type .llcrypt registered
[21524.672365] libcfs: HW NUMA nodes: 1, HW CPU cores: 2, npartitions: 2
[21525.468577] Lustre: Lustre: Build Version: 2.16.50_134_g3a7cdc0
[21525.521234] LNet: Added LNI 10.240.22.180@tcp [8/256/0/180]
[21525.522532] LNet: Accept all, port 7988
[21527.053411] Key type lgssc registered
[21527.206932] Lustre: Echo OBD driver; http://www.lustre.org/
[21549.934948] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre' ' /proc/mounts);
[21549.934948] mpts=$(mount | grep -c /mnt/lustre' ');
[21549.934948] if [ $running -ne $mpts ]; then
[21549.934948]     echo $(hostname) env are INSANE!;
[21549.934948]     exit 1;
[21549.934948] fi
[21550.726500] Lustre: DEBUG MARKER: running=$(grep -c /mnt/lustre2' ' /proc/mounts);
[21550.726500] mpts=$(mount | grep -c /mnt/lustre2' ');
[21550.726500] if [ $running -ne $mpts ]; then
[21550.726500]     echo $(hostname) env are INSANE!;
[21550.726500]     exit 1;
[21550.726500] fi
[21603.637744] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21603.648471] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre
[21604.686292] Lustre: Mounted lustre-client
[21605.031862] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[21605.714148] Lustre: DEBUG MARKER: 
[21605.714148] running=$(mount | grep -c /mnt/lustre' ');
[21605.714148] rc=0;
[21605.714148] if [ $running -eq 0 ] ; then
[21605.714148] 	mkdir -p /mnt/lustre;
[21605.714148] 	mount -t lustre  -o user_xattr,flock 10.240.22.189@tcp:/lustre /mnt/lustre;
[21605.714148] 	rc=$?;
[21605.714148] fi;
[21605.714148] exit $rc
[21607.439592] Lustre: DEBUG MARKER: mount | grep /mnt/lustre' '
[21609.119931] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[21609.136411] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var
[21626.152816] Lustre: 1025784:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1734413473/real 0]  req@ffff988d0f069380 x1818664253788288/t0(0) o400->lustre-MDT0000-mdc-ffff988c12222800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734413489 ref 2 fl Rpc:XNr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[21626.158280] Lustre: lustre-MDT0000-mdc-ffff988c12222800: Connection to lustre-MDT0000 (at 10.240.22.189@tcp) was lost; in progress operations using this service will wait for recovery to complete
[21626.161729] LustreError: MGC10.240.22.189@tcp: Connection to MGS (at 10.240.22.189@tcp) was lost; in progress operations using this service will fail
[21630.152797] Lustre: 1025785:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1734413478/real 0]  req@ffff988d0f068680 x1818664253789440/t0(0) o400->lustre-MDT0000-mdc-ffff988c12222800@10.240.22.189@tcp:12/10 lens 224/224 e 0 to 1 dl 1734413494 ref 2 fl Rpc:XNr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[21630.158266] Lustre: 1025785:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 8 previous similar messages
[21635.272786] Lustre: 1025785:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1734413483/real 0]  req@ffff988d08e07740 x1818664253790720/t0(0) o400->lustre-OST0000-osc-ffff988c12222800@10.240.22.182@tcp:28/4 lens 224/224 e 0 to 1 dl 1734413499 ref 2 fl Rpc:XNr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[21635.278184] Lustre: 1025785:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 7 previous similar messages
[21641.512736] Lustre: 1025784:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1734413489/real 0]  req@ffff988d08e05d40 x1818664253791872/t0(0) o400->lustre-OST0000-osc-ffff988c12222800@10.240.22.182@tcp:28/4 lens 224/224 e 0 to 1 dl 1734413505 ref 2 fl Rpc:XNr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0
[21641.518253] Lustre: 1025784:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 7 previous similar messages
[21791.783948] nfs: server 10.240.16.204 not responding, still trying
[21819.431773] nfs: server 10.240.16.204 not responding, still trying
[21819.433120] nfs: server 10.240.16.204 not responding, still trying
[21825.575673] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[21825.575783] Lustre: 1025784:0:(client.c:2358:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1734413489/real 1734413689]  req@ffff988d08e070c0 x1818664253791616/t0(0) o400->MGC10.240.22.189@tcp@10.240.22.189@tcp:26/25 lens 224/224 e 0 to 1 dl 1734413505 ref 1 fl Rpc:EeXNQU/200/ffffffff rc -5/-1 job:'' uid:0 gid:0
[21825.582667] Lustre: 1025784:0:(client.c:2358:ptlrpc_expire_one_request()) Skipped 15 previous similar messages
[21825.584775] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[21826.599641] LNetError: Unexpected error -2 connecting to 10.240.22.182@tcp at host 10.240.22.182:7988
[21826.601428] LNetError: Skipped 1 previous similar message
[21828.647678] LNetError: Unexpected error -2 connecting to 10.240.22.182@tcp at host 10.240.22.182:7988
[21828.649560] LNetError: Skipped 2 previous similar messages
[21832.743648] LNetError: Unexpected error -2 connecting to 10.240.22.182@tcp at host 10.240.22.182:7988
[21832.745460] LNetError: Skipped 1 previous similar message
[21840.935668] LNetError: Unexpected error -2 connecting to 10.240.22.182@tcp at host 10.240.22.182:7988
[21840.937522] LNetError: Skipped 1 previous similar message
[21847.079510] nfs: server 10.240.16.204 not responding, timed out
[21856.295507] LNetError: Unexpected error -2 connecting to 10.240.22.182@tcp at host 10.240.22.182:7988
[21856.297560] LNetError: Skipped 1 previous similar message
[21885.991276] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[21888.039300] LNetError: Unexpected error -2 connecting to 10.240.22.182@tcp at host 10.240.22.182:7988
[21888.041204] LNetError: Skipped 1 previous similar message
[21948.455069] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[21948.456883] LNetError: Skipped 1 previous similar message
[22005.798570] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[22027.686474] nfs: server 10.240.16.204 not responding, timed out
[22032.678418] nfs: server 10.240.16.204 not responding, timed out
[22071.334452] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[22071.336278] LNetError: Skipped 3 previous similar messages
[22186.021513] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[22208.805394] nfs: server 10.240.16.204 not responding, timed out
[22217.957334] nfs: server 10.240.16.204 not responding, timed out
[22255.653391] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[22255.655236] LNetError: Skipped 5 previous similar messages
[22389.412328] nfs: server 10.240.16.204 not responding, timed out
[22394.404284] nfs: server 10.240.16.204 not responding, timed out
[22403.428222] nfs: server 10.240.16.204 not responding, timed out
[22425.636082] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[22562.851524] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[22562.853341] LNetError: Skipped 9 previous similar messages
[22569.891238] nfs: server 10.240.16.204 not responding, timed out
[22579.939172] nfs: server 10.240.16.204 not responding, timed out
[22588.963110] nfs: server 10.240.16.204 not responding, timed out
[22725.666302] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[22749.986176] nfs: server 10.240.16.204 not responding, timed out
[22756.130197] nfs: server 10.240.16.204 not responding, timed out
[22765.154084] nfs: server 10.240.16.204 not responding, timed out
[22774.178046] nfs: server 10.240.16.204 not responding, timed out
[22930.593114] nfs: server 10.240.16.204 not responding, timed out
[22935.585061] nfs: server 10.240.16.204 not responding, timed out
[22941.601015] nfs: server 10.240.16.204 not responding, timed out
[22950.624980] nfs: server 10.240.16.204 not responding, timed out
[22959.680901] nfs: server 10.240.16.204 not responding, timed out
[23025.696540] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[23111.072021] nfs: server 10.240.16.204 not responding, timed out
[23115.808124] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[23115.809947] LNetError: Skipped 17 previous similar messages
[23121.183959] nfs: server 10.240.16.204 not responding, timed out
[23127.199929] nfs: server 10.240.16.204 not responding, timed out
[23136.223881] nfs: server 10.240.16.204 not responding, timed out
[23145.247802] nfs: server 10.240.16.204 not responding, timed out
[23292.446947] nfs: server 10.240.16.204 not responding, timed out
[23298.590902] nfs: server 10.240.16.204 not responding, timed out
[23307.614865] nfs: server 10.240.16.204 not responding, timed out
[23313.630807] nfs: server 10.240.16.204 not responding, timed out
[23322.654812] nfs: server 10.240.16.204 not responding, timed out
[23325.726762] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[23331.678723] nfs: server 10.240.16.204 not responding, timed out
[23473.053862] nfs: server 10.240.16.204 not responding, timed out
[23478.045840] nfs: server 10.240.16.204 not responding, timed out
[23484.061792] nfs: server 10.240.16.204 not responding, timed out
[23493.085736] nfs: server 10.240.16.204 not responding, timed out
[23499.101707] nfs: server 10.240.16.204 not responding, timed out
[23508.125662] nfs: server 10.240.16.204 not responding, timed out
[23517.149606] nfs: server 10.240.16.204 not responding, timed out
[23625.756965] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[23653.532833] nfs: server 10.240.16.204 not responding, timed out
[23663.580749] nfs: server 10.240.16.204 not responding, timed out
[23669.596706] nfs: server 10.240.16.204 not responding, timed out
[23678.620659] nfs: server 10.240.16.204 not responding, timed out
[23684.636607] nfs: server 10.240.16.204 not responding, timed out
[23693.660551] nfs: server 10.240.16.204 not responding, timed out
[23702.684528] nfs: server 10.240.16.204 not responding, timed out
[23730.204583] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[23730.206404] LNetError: Skipped 19 previous similar messages
[23834.011722] nfs: server 10.240.16.204 not responding, timed out
[23841.051698] nfs: server 10.240.16.204 not responding, timed out
[23850.075625] nfs: server 10.240.16.204 not responding, timed out
[23856.091588] nfs: server 10.240.16.204 not responding, timed out
[23865.115555] nfs: server 10.240.16.204 not responding, timed out
[23871.131514] nfs: server 10.240.16.204 not responding, timed out
[23880.155439] nfs: server 10.240.16.204 not responding, timed out
[23888.155419] nfs: server 10.240.16.204 not responding, timed out
[23925.787172] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[24015.514660] nfs: server 10.240.16.204 not responding, timed out
[24021.530610] nfs: server 10.240.16.204 not responding, timed out
[24027.546581] nfs: server 10.240.16.204 not responding, timed out
[24036.570540] nfs: server 10.240.16.204 not responding, timed out
[24042.586491] nfs: server 10.240.16.204 not responding, timed out
[24051.610433] nfs: server 10.240.16.204 not responding, timed out
[24056.602418] nfs: server 10.240.16.204 not responding, timed out
[24066.650371] nfs: server 10.240.16.204 not responding, timed out
[24075.674294] nfs: server 10.240.16.204 not responding, timed out
[24195.993576] nfs: server 10.240.16.204 not responding, timed out
[24202.009545] nfs: server 10.240.16.204 not responding, timed out
[24208.025511] nfs: server 10.240.16.204 not responding, timed out
[24213.017478] nfs: server 10.240.16.204 not responding, timed out
[24223.065410] nfs: server 10.240.16.204 not responding, timed out
[24225.817391] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[24229.081375] nfs: server 10.240.16.204 not responding, timed out
[24244.121313] nfs: server 10.240.16.204 not responding, timed out
[24253.145233] nfs: server 10.240.16.204 not responding, timed out
[24262.169184] nfs: server 10.240.16.204 not responding, timed out
[24344.600954] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[24344.602801] LNetError: Skipped 17 previous similar messages
[24376.472511] nfs: server 10.240.16.204 not responding, timed out
[24382.488473] nfs: server 10.240.16.204 not responding, timed out
[24387.480451] nfs: server 10.240.16.204 not responding, timed out
[24400.536365] nfs: server 10.240.16.204 not responding, timed out
[24409.560302] nfs: server 10.240.16.204 not responding, timed out
[24415.576293] nfs: server 10.240.16.204 not responding, timed out
[24430.616186] nfs: server 10.240.16.204 not responding, timed out
[24439.640134] nfs: server 10.240.16.204 not responding, timed out
[24447.640095] nfs: server 10.240.16.204 not responding, timed out
[24525.847623] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[24556.951430] nfs: server 10.240.16.204 not responding, timed out
[24562.967411] nfs: server 10.240.16.204 not responding, timed out
[24574.999378] nfs: server 10.240.16.204 not responding, timed out
[24587.031335] nfs: server 10.240.16.204 not responding, timed out
[24596.055230] nfs: server 10.240.16.204 not responding, timed out
[24602.071165] nfs: server 10.240.16.204 not responding, timed out
[24616.087096] nfs: server 10.240.16.204 not responding, timed out
[24626.135047] nfs: server 10.240.16.204 not responding, timed out
[24635.159023] nfs: server 10.240.16.204 not responding, timed out
[24737.430397] nfs: server 10.240.16.204 not responding, timed out
[24743.446343] nfs: server 10.240.16.204 not responding, timed out
[24749.462300] nfs: server 10.240.16.204 not responding, timed out
[24761.494235] nfs: server 10.240.16.204 not responding, timed out
[24772.502222] nfs: server 10.240.16.204 not responding, timed out
[24782.550114] nfs: server 10.240.16.204 not responding, timed out
[24788.566080] nfs: server 10.240.16.204 not responding, timed out
[24803.605985] nfs: server 10.240.16.204 not responding, timed out
[24812.629954] nfs: server 10.240.16.204 not responding, timed out
[24820.629868] nfs: server 10.240.16.204 not responding, timed out
[24825.877901] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[24917.909308] nfs: server 10.240.16.204 not responding, timed out
[24923.925257] nfs: server 10.240.16.204 not responding, timed out
[24929.941234] nfs: server 10.240.16.204 not responding, timed out
[24935.957197] nfs: server 10.240.16.204 not responding, timed out
[24947.989133] nfs: server 10.240.16.204 not responding, timed out
[24958.997269] LNetError: Unexpected error -2 connecting to 10.240.22.182@tcp at host 10.240.22.182:7988
[24958.999074] LNetError: Skipped 19 previous similar messages
[24960.021070] nfs: server 10.240.16.204 not responding, timed out
[24969.045009] nfs: server 10.240.16.204 not responding, timed out
[24975.060992] nfs: server 10.240.16.204 not responding, timed out
[24989.076889] nfs: server 10.240.16.204 not responding, timed out
[24999.124816] nfs: server 10.240.16.204 not responding, timed out
[25008.148755] nfs: server 10.240.16.204 not responding, timed out
[25098.388226] nfs: server 10.240.16.204 not responding, timed out
[25104.404183] nfs: server 10.240.16.204 not responding, timed out
[25110.420153] nfs: server 10.240.16.204 not responding, timed out
[25116.436133] nfs: server 10.240.16.204 not responding, timed out
[25122.452085] nfs: server 10.240.16.204 not responding, timed out
[25125.908048] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[25133.460043] nfs: server 10.240.16.204 not responding, timed out
[25146.515941] nfs: server 10.240.16.204 not responding, timed out
[25155.539932] nfs: server 10.240.16.204 not responding, timed out
[25161.555848] nfs: server 10.240.16.204 not responding, timed out
[25176.595773] nfs: server 10.240.16.204 not responding, timed out
[25185.619704] nfs: server 10.240.16.204 not responding, timed out
[25194.643654] nfs: server 10.240.16.204 not responding, timed out
[25278.867144] nfs: server 10.240.16.204 not responding, timed out
[25284.883128] nfs: server 10.240.16.204 not responding, timed out
[25289.875114] nfs: server 10.240.16.204 not responding, timed out
[25296.915073] nfs: server 10.240.16.204 not responding, timed out
[25302.931022] nfs: server 10.240.16.204 not responding, timed out
[25308.946984] nfs: server 10.240.16.204 not responding, timed out
[25319.954995] nfs: server 10.240.16.204 not responding, timed out
[25333.010911] nfs: server 10.240.16.204 not responding, timed out
[25342.034790] nfs: server 10.240.16.204 not responding, timed out
[25348.050758] nfs: server 10.240.16.204 not responding, timed out
[25363.090678] nfs: server 10.240.16.204 not responding, timed out
[25371.090635] nfs: server 10.240.16.204 not responding, timed out
[25381.138584] nfs: server 10.240.16.204 not responding, timed out
[25425.938300] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[25459.346081] nfs: server 10.240.16.204 not responding, timed out
[25465.362047] nfs: server 10.240.16.204 not responding, timed out
[25477.393983] nfs: server 10.240.16.204 not responding, timed out
[25483.409950] nfs: server 10.240.16.204 not responding, timed out
[25489.425928] nfs: server 10.240.16.204 not responding, timed out
[25495.441881] nfs: server 10.240.16.204 not responding, timed out
[25507.473802] nfs: server 10.240.16.204 not responding, timed out
[25519.505744] nfs: server 10.240.16.204 not responding, timed out
[25528.529685] nfs: server 10.240.16.204 not responding, timed out
[25533.521717] nfs: server 10.240.16.204 not responding, timed out
[25549.585568] nfs: server 10.240.16.204 not responding, timed out
[25558.609500] nfs: server 10.240.16.204 not responding, timed out
[25567.633446] nfs: server 10.240.16.204 not responding, timed out
[25573.393555] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[25573.395420] LNetError: Skipped 19 previous similar messages
[25639.825011] nfs: server 10.240.16.204 not responding, timed out
[25645.841004] nfs: server 10.240.16.204 not responding, timed out
[25651.856960] nfs: server 10.240.16.204 not responding, timed out
[25663.888876] nfs: server 10.240.16.204 not responding, timed out
[25669.904845] nfs: server 10.240.16.204 not responding, timed out
[25675.920865] nfs: server 10.240.16.204 not responding, timed out
[25681.936778] nfs: server 10.240.16.204 not responding, timed out
[25693.968700] nfs: server 10.240.16.204 not responding, timed out
[25704.976637] nfs: server 10.240.16.204 not responding, timed out
[25721.040595] nfs: server 10.240.16.204 not responding, timed out
[25725.968510] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[25736.080441] nfs: server 10.240.16.204 not responding, timed out
[25745.104407] nfs: server 10.240.16.204 not responding, timed out
[25753.104367] nfs: server 10.240.16.204 not responding, timed out
[25820.303955] nfs: server 10.240.16.204 not responding, timed out
[25826.319983] nfs: server 10.240.16.204 not responding, timed out
[25832.335915] nfs: server 10.240.16.204 not responding, timed out
[25837.327848] nfs: server 10.240.16.204 not responding, timed out
[25850.383798] nfs: server 10.240.16.204 not responding, timed out
[25856.399732] nfs: server 10.240.16.204 not responding, timed out
[25862.415701] nfs: server 10.240.16.204 not responding, timed out
[25868.431675] nfs: server 10.240.16.204 not responding, timed out
[25880.463590] nfs: server 10.240.16.204 not responding, timed out
[25892.495518] nfs: server 10.240.16.204 not responding, timed out
[25907.535468] nfs: server 10.240.16.204 not responding, timed out
[25922.575391] nfs: server 10.240.16.204 not responding, timed out
[25930.575293] nfs: server 10.240.16.204 not responding, timed out
[25940.623236] nfs: server 10.240.16.204 not responding, timed out
[26000.782879] nfs: server 10.240.16.204 not responding, timed out
[26006.798887] nfs: server 10.240.16.204 not responding, timed out
[26012.814850] nfs: server 10.240.16.204 not responding, timed out
[26024.846746] nfs: server 10.240.16.204 not responding, timed out
[26025.998743] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[26036.878681] nfs: server 10.240.16.204 not responding, timed out
[26042.894653] nfs: server 10.240.16.204 not responding, timed out
[26048.910617] nfs: server 10.240.16.204 not responding, timed out
[26054.926551] nfs: server 10.240.16.204 not responding, timed out
[26066.958522] nfs: server 10.240.16.204 not responding, timed out
[26078.990414] nfs: server 10.240.16.204 not responding, timed out
[26093.006398] nfs: server 10.240.16.204 not responding, timed out
[26109.070235] nfs: server 10.240.16.204 not responding, timed out
[26117.070198] nfs: server 10.240.16.204 not responding, timed out
[26126.094163] nfs: server 10.240.16.204 not responding, timed out
[26181.261808] nfs: server 10.240.16.204 not responding, timed out
[26187.277783] nfs: server 10.240.16.204 not responding, timed out
[26187.790035] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[26187.791839] LNetError: Skipped 19 previous similar messages
[26193.293729] nfs: server 10.240.16.204 not responding, timed out
[26199.309711] nfs: server 10.240.16.204 not responding, timed out
[26211.341689] nfs: server 10.240.16.204 not responding, timed out
[26223.373582] nfs: server 10.240.16.204 not responding, timed out
[26229.389519] nfs: server 10.240.16.204 not responding, timed out
[26235.405487] nfs: server 10.240.16.204 not responding, timed out
[26241.421457] nfs: server 10.240.16.204 not responding, timed out
[26253.453378] nfs: server 10.240.16.204 not responding, timed out
[26265.485344] nfs: server 10.240.16.204 not responding, timed out
[26280.525227] nfs: server 10.240.16.204 not responding, timed out
[26294.541145] nfs: server 10.240.16.204 not responding, timed out
[26304.589083] nfs: server 10.240.16.204 not responding, timed out
[26312.589030] nfs: server 10.240.16.204 not responding, timed out
[26326.028957] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[26361.740751] nfs: server 10.240.16.204 not responding, timed out
[26367.756712] nfs: server 10.240.16.204 not responding, timed out
[26372.748673] nfs: server 10.240.16.204 not responding, timed out
[26379.788643] nfs: server 10.240.16.204 not responding, timed out
[26385.804603] nfs: server 10.240.16.204 not responding, timed out
[26396.812545] nfs: server 10.240.16.204 not responding, timed out
[26409.868448] nfs: server 10.240.16.204 not responding, timed out
[26415.884418] nfs: server 10.240.16.204 not responding, timed out
[26421.900395] nfs: server 10.240.16.204 not responding, timed out
[26427.916343] nfs: server 10.240.16.204 not responding, timed out
[26438.924368] nfs: server 10.240.16.204 not responding, timed out
[26451.980233] nfs: server 10.240.16.204 not responding, timed out
[26467.020154] nfs: server 10.240.16.204 not responding, timed out
[26481.036030] nfs: server 10.240.16.204 not responding, timed out
[26491.084008] nfs: server 10.240.16.204 not responding, timed out
[26500.107932] nfs: server 10.240.16.204 not responding, timed out
[26542.219696] nfs: server 10.240.16.204 not responding, timed out
[26548.235646] nfs: server 10.240.16.204 not responding, timed out
[26560.267582] nfs: server 10.240.16.204 not responding, timed out
[26566.283519] nfs: server 10.240.16.204 not responding, timed out
[26572.299517] nfs: server 10.240.16.204 not responding, timed out
[26584.331417] nfs: server 10.240.16.204 not responding, timed out
[26596.363345] nfs: server 10.240.16.204 not responding, timed out
[26602.379308] nfs: server 10.240.16.204 not responding, timed out
[26608.395274] nfs: server 10.240.16.204 not responding, timed out
[26626.059157] LNet: 2 peer NIs in recovery (showing 2): 10.240.22.182@tcp, 10.240.22.189@tcp
[26637.451112] nfs: server 10.240.16.204 not responding, timed out
[26637.452411] nfs: server 10.240.16.204 not responding, timed out
[26653.515002] nfs: server 10.240.16.204 not responding, timed out
[26653.516326] nfs: server 10.240.16.204 not responding, timed out
[26677.578856] nfs: server 10.240.16.204 not responding, timed out
[26677.580170] nfs: server 10.240.16.204 not responding, timed out
[26722.698584] nfs: server 10.240.16.204 not responding, timed out
[26722.699938] nfs: server 10.240.16.204 not responding, timed out
[26746.762592] nfs: server 10.240.16.204 not responding, timed out
[26752.778419] nfs: server 10.240.16.204 not responding, timed out
[26752.779664] nfs: server 10.240.16.204 not responding, timed out
[26770.826307] nfs: server 10.240.16.204 not responding, timed out
[26770.827604] nfs: server 10.240.16.204 not responding, timed out
[26781.834237] nfs: server 10.240.16.204 not responding, timed out
[26781.835515] nfs: server 10.240.16.204 not responding, timed out
[26788.874197] nfs: server 10.240.16.204 not responding, timed out
[26794.890195] nfs: server 10.240.16.204 not responding, timed out
[26802.186259] LNetError: Unexpected error -2 connecting to 10.240.22.189@tcp at host 10.240.22.189:7988
[26802.188057] LNetError: Skipped 19 previous similar messages
[26824.969981] nfs: server 10.240.16.204 not responding, timed out
[26840.009907] nfs: server 10.240.16.204 not responding, timed out
[26864.073802] nfs: server 10.240.16.204 not responding, timed out
[26903.177535] nfs: server 10.240.16.204 not responding, timed out
[26909.193498] nfs: server 10.240.16.204 not responding, timed out