Sep 10 12:14:28 spoon17.ccs.ornl.gov kernel: imklog 5.8.10, log source = /proc/kmsg started. Sep 10 12:17:29 spoon17.ccs.ornl.gov kernel: [ 291.090597] LNet: HW CPU cores: 8, npartitions: 2 Sep 10 12:17:29 spoon17.ccs.ornl.gov kernel: [ 291.113609] alg: No test for adler32 (adler32-zlib) Sep 10 12:17:29 spoon17.ccs.ornl.gov kernel: [ 291.118698] alg: No test for crc32 (crc32-table) Sep 10 12:17:37 spoon17.ccs.ornl.gov kernel: [ 299.315650] Lustre: Lustre: Build Version: 2.7.59-g46ebfdd-CHANGED-2.6.32-504.30.3.el6.head.x86_64 Sep 10 12:17:38 spoon17.ccs.ornl.gov kernel: [ 300.038619] LNet: Added LNI 10.37.248.19@o2ib1 [63/2560/0/180] Sep 10 12:18:32 spoon17.ccs.ornl.gov kernel: [ 353.609965] Lustre: 19836:0:(gss_svc_upcall.c:1129:gss_init_svc_upcall()) Init channel is not opened by lsvcgssd, following request might be dropped until lsvcgssd is active Sep 10 12:18:32 spoon17.ccs.ornl.gov kernel: [ 353.625498] Lustre: 19836:0:(gss_mech_switch.c:72:lgss_mech_register()) Register gssnull mechanism Sep 10 12:18:32 spoon17.ccs.ornl.gov kernel: [ 353.976183] Lustre: Echo OBD driver; http://www.lustre.org/ Sep 10 12:18:35 spoon17.ccs.ornl.gov kernel: [ 356.624872] Lustre: DEBUG MARKER: grep -c /tmp/mds1' ' /proc/mounts Sep 10 12:18:35 spoon17.ccs.ornl.gov kernel: [ 356.686048] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:18:35 spoon17.ccs.ornl.gov kernel: [ 356.946559] Lustre: DEBUG MARKER: mkfs.lustre --mgs --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reformat /tmp/lustre-mdt1 Sep 10 12:18:35 spoon17.ccs.ornl.gov kernel: [ 357.018077] zunicode: module license 'CDDL' taints kernel. Sep 10 12:18:35 spoon17.ccs.ornl.gov kernel: [ 357.023646] Disabling lock debugging due to kernel taint Sep 10 12:18:35 spoon17.ccs.ornl.gov kernel: [ 357.084803] SPL: Loaded module v0.6.4.2-1 Sep 10 12:18:35 spoon17.ccs.ornl.gov kernel: [ 357.352817] ZFS: Loaded module v0.6.4.2-1, ZFS pool version 5000, ZFS filesystem version 5 Sep 10 12:18:36 spoon17.ccs.ornl.gov kernel: [ 357.560960] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:36 spoon17.ccs.ornl.gov kernel: [ 357.995998] Lustre: DEBUG MARKER: grep -c /tmp/mds2' ' /proc/mounts Sep 10 12:18:36 spoon17.ccs.ornl.gov kernel: [ 358.058403] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:18:36 spoon17.ccs.ornl.gov kernel: [ 358.134998] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=1 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:18:36 spoon17.ccs.ornl.gov kernel: [ 358.380440] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:37 spoon17.ccs.ornl.gov kernel: [ 358.806157] Lustre: DEBUG MARKER: grep -c /tmp/mds3' ' /proc/mounts Sep 10 12:18:37 spoon17.ccs.ornl.gov kernel: [ 358.869589] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:18:37 spoon17.ccs.ornl.gov kernel: [ 358.945924] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=2 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:18:37 spoon17.ccs.ornl.gov kernel: [ 359.166658] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:38 spoon17.ccs.ornl.gov kernel: [ 359.592854] Lustre: DEBUG MARKER: grep -c /tmp/mds4' ' /proc/mounts Sep 10 12:18:38 spoon17.ccs.ornl.gov kernel: [ 359.654105] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:18:38 spoon17.ccs.ornl.gov kernel: [ 359.729900] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=3 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:18:38 spoon17.ccs.ornl.gov kernel: [ 359.971516] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:38 spoon17.ccs.ornl.gov kernel: [ 360.299922] Lustre: DEBUG MARKER: grep -c /tmp/ost1' ' /proc/mounts Sep 10 12:18:38 spoon17.ccs.ornl.gov kernel: [ 360.359993] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:18:38 spoon17.ccs.ornl.gov kernel: [ 360.437464] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --ost --index=0 --param=sys.timeout=20 --backfstype=ldiskfs --device-size=400000 --reformat /tmp/lustre-ost1 Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 360.615051] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 360.942637] Lustre: DEBUG MARKER: grep -c /tmp/ost2' ' /proc/mounts Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 361.005581] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 361.082713] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --ost --index=1 --param=sys.timeout=20 --backfstype=ldiskfs --device-size=400000 --reformat /tmp/lustre-ost2 Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 361.245289] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 361.347918] Lustre: DEBUG MARKER: running=$(grep -c /tmp/ost1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/ost1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 361.430691] Lustre: DEBUG MARKER: running=$(grep -c /tmp/ost2' ' /proc/mounts); mpts=$(mount | grep -c /tmp/ost2' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:39 spoon17.ccs.ornl.gov kernel: [ 361.512145] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:40 spoon17.ccs.ornl.gov kernel: [ 361.596414] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds2' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds2' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:40 spoon17.ccs.ornl.gov kernel: [ 361.678376] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds3' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds3' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:40 spoon17.ccs.ornl.gov kernel: [ 361.758241] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds4' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds4' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:40 spoon17.ccs.ornl.gov kernel: [ 361.983943] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:40 spoon17.ccs.ornl.gov kernel: [ 362.025057] Lustre: DEBUG MARKER: running=$(grep -c /lustre/lustre' ' /proc/mounts); mpts=$(mount | grep -c /lustre/lustre' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:40 spoon17.ccs.ornl.gov kernel: [ 362.064188] Lustre: DEBUG MARKER: running=$(grep -c /lustre/testfs' ' /proc/mounts); mpts=$(mount | grep -c /lustre/testfs' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.552949] Lustre: DEBUG MARKER: mkdir -p /tmp/mds1 Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.621391] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt1 Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.689913] Lustre: DEBUG MARKER: mkdir -p /tmp/mds1; mount -t lustre -o loop /tmp/lustre-mdt1 /tmp/mds1 Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.761592] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.773856] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.903742] Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.917551] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.972905] Lustre: lustre-MDT0000: new disk, initializing Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 363.993305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 364.129452] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 364.197769] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt1 2>/dev/null Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 364.263999] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 364.415948] Lustre: DEBUG MARKER: mkdir -p /tmp/mds2 Sep 10 12:18:42 spoon17.ccs.ornl.gov kernel: [ 364.483783] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt2 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.552197] Lustre: DEBUG MARKER: mkdir -p /tmp/mds2; mount -t lustre -o loop /tmp/lustre-mdt2 /tmp/mds2 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.619206] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.640425] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.723090] Lustre: Setting parameter lustre-MDT0001-mdtlov.lov.stripesize in log lustre-MDT0001 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.732053] Lustre: Skipped 4 previous similar messages Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.741819] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.749391] Lustre: Skipped 1 previous similar message Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.802156] Lustre: lustre-MDT0001: new disk, initializing Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.829686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400):1:mdt Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.840611] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400):1:mdt] Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 364.970938] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 365.040914] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt2 2>/dev/null Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 365.103987] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 365.255634] Lustre: DEBUG MARKER: mkdir -p /tmp/mds3 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 365.322986] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt3 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 365.391397] Lustre: DEBUG MARKER: mkdir -p /tmp/mds3; mount -t lustre -o loop /tmp/lustre-mdt3 /tmp/mds3 Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 365.460026] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:43 spoon17.ccs.ornl.gov kernel: [ 365.481932] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 365.649114] Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400):2:mdt] Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 365.777157] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 365.846973] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt3 2>/dev/null Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 365.910307] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.065443] Lustre: DEBUG MARKER: mkdir -p /tmp/mds4 Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.133776] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt4 Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.201481] Lustre: DEBUG MARKER: mkdir -p /tmp/mds4; mount -t lustre -o loop /tmp/lustre-mdt4 /tmp/mds4 Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.269762] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.292027] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.378707] Lustre: Setting parameter lustre-MDT0003-mdtlov.lov.stripesize in log lustre-MDT0003 Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.387737] Lustre: Skipped 9 previous similar messages Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.397182] Lustre: srv-lustre-MDT0003: No data found on store. Initialize space Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.404767] Lustre: Skipped 1 previous similar message Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.459054] Lustre: lustre-MDT0003: new disk, initializing Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.464656] Lustre: Skipped 1 previous similar message Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.506457] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400):3:mdt Sep 10 12:18:44 spoon17.ccs.ornl.gov kernel: [ 366.517334] Lustre: Skipped 1 previous similar message Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 366.634008] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 366.703160] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt4 2>/dev/null Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 366.768709] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 366.921766] Lustre: DEBUG MARKER: mkdir -p /tmp/ost1 Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 366.988779] Lustre: DEBUG MARKER: test -b /tmp/lustre-ost1 Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 367.056817] Lustre: DEBUG MARKER: mkdir -p /tmp/ost1; mount -t lustre -o loop /tmp/lustre-ost1 /tmp/ost1 Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 367.128772] LDISKFS-fs (loop4): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 367.148181] LDISKFS-fs (loop4): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:45 spoon17.ccs.ornl.gov kernel: [ 367.448143] Lustre: DEBUG MARKER: e2label /tmp/lustre-ost1 2>/dev/null Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 367.599885] Lustre: DEBUG MARKER: mkdir -p /tmp/ost2 Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 367.668135] Lustre: DEBUG MARKER: test -b /tmp/lustre-ost2 Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 367.736860] Lustre: DEBUG MARKER: mkdir -p /tmp/ost2; mount -t lustre -o loop /tmp/lustre-ost2 /tmp/ost2 Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 367.803324] LDISKFS-fs (loop5): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 367.832115] LDISKFS-fs (loop5): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 368.106048] Lustre: DEBUG MARKER: e2label /tmp/lustre-ost2 2>/dev/null Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 368.171507] Lustre: DEBUG MARKER: mkdir -p /lustre/lustre Sep 10 12:18:46 spoon17.ccs.ornl.gov kernel: [ 368.190317] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.37.248.19@o2ib1:/lustre /lustre/lustre Sep 10 12:18:51 spoon17.ccs.ornl.gov kernel: [ 372.621729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400):0:ost Sep 10 12:18:51 spoon17.ccs.ornl.gov kernel: [ 372.632709] Lustre: cli-ffff8800cecaf044-su: Allocated super-sequence [0x0000000300000400-0x0000000340000400):0:ost] Sep 10 12:18:51 spoon17.ccs.ornl.gov kernel: [ 372.643406] Lustre: Skipped 2 previous similar messages Sep 10 12:18:56 spoon17.ccs.ornl.gov kernel: [ 377.762251] Lustre: Mounted lustre-client Sep 10 12:18:56 spoon17.ccs.ornl.gov kernel: [ 377.797171] Lustre: DEBUG MARKER: running=$(mount | grep -c /lustre/lustre' '); rc=0; if [ $running -eq 0 ] ; then mkdir -p /lustre/lustre; mount -t lustre -o user_xattr,flock 10.37.248.19@o2ib1:/lustre /lustre/lustre; rc=$?; fi; exit $rc Sep 10 12:18:56 spoon17.ccs.ornl.gov kernel: [ 377.835290] Lustre: DEBUG MARKER: mount | grep /lustre/lustre' ' Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 378.987202] Lustre: DEBUG MARKER: lctl get_param -n timeout Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.026238] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20 Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.041444] Lustre: DEBUG MARKER: Using TIMEOUT=20 Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.092530] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.118695] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.153522] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.176004] Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param lustre.sys.jobid_var='procname_uid' Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.202413] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:18:57 spoon17.ccs.ornl.gov kernel: [ 379.223885] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:18:58 spoon17.ccs.ornl.gov kernel: [ 380.248822] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:18:59 spoon17.ccs.ornl.gov kernel: [ 381.273438] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:19:00 spoon17.ccs.ornl.gov kernel: [ 382.297168] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:20:02 spoon17.ccs.ornl.gov kernel: [ 444.188264] Lustre: DEBUG MARKER: Using TIMEOUT=20 Sep 10 12:20:03 spoon17.ccs.ornl.gov kernel: [ 444.668857] Lustre: DEBUG MARKER: excepting tests: 76 42a 42b 42c 42d 45 51d 68b Sep 10 12:20:03 spoon17.ccs.ornl.gov kernel: [ 444.707324] Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24D 27m 64b 68 71 77f 78 115 124b Sep 10 12:20:04 spoon17.ccs.ornl.gov kernel: [ 445.828670] Lustre: DEBUG MARKER: == sanity test 300a: basic striped dir sanity test == 12:20:04 (1441902004) Sep 10 12:20:04 spoon17.ccs.ornl.gov kernel: [ 446.392444] Lustre: DEBUG MARKER: == sanity test 300b: check ctime/mtime for striped dir == 12:20:04 (1441902004) Sep 10 12:20:25 spoon17.ccs.ornl.gov kernel: [ 466.968315] Lustre: DEBUG MARKER: == sanity test 300c: chown Sep 10 12:21:06 spoon17.ccs.ornl.gov kernel: [ 508.383643] Lustre: DEBUG MARKER: == sanity test 300d: check default stripe under striped directory == 12:21:06 (1441902066) Sep 10 12:21:07 spoon17.ccs.ornl.gov kernel: [ 509.183017] Lustre: DEBUG MARKER: == sanity test 300e: check rename under striped directory == 12:21:07 (1441902067) Sep 10 12:21:08 spoon17.ccs.ornl.gov kernel: [ 509.621410] Lustre: DEBUG MARKER: == sanity test 300f: check rename cross striped directory == 12:21:08 (1441902068) Sep 10 12:21:08 spoon17.ccs.ornl.gov kernel: [ 510.051847] Lustre: DEBUG MARKER: == sanity test 300g: check default striped directory for normal directory == 12:21:08 (1441902068) Sep 10 12:21:09 spoon17.ccs.ornl.gov kernel: [ 510.784902] Lustre: DEBUG MARKER: sanity test_300g: @@@@@@ FAIL: stripe count 1 != 0 for /lustre/lustre/d300g.sanity/normal_dir/test1 Sep 10 12:21:09 spoon17.ccs.ornl.gov kernel: [ 511.513556] Lustre: DEBUG MARKER: == sanity test 300h: check default striped directory for striped directory == 12:21:09 (1441902069) Sep 10 12:21:10 spoon17.ccs.ornl.gov kernel: [ 512.302516] Lustre: DEBUG MARKER: sanity test_300h: @@@@@@ FAIL: stripe count 1 != 0 for /lustre/lustre/d300h.sanity/striped_dir/test1 Sep 10 12:21:11 spoon17.ccs.ornl.gov kernel: [ 512.930827] Lustre: DEBUG MARKER: == sanity test 300i: client handle unknown hash type striped directory == 12:21:11 (1441902071) Sep 10 12:21:11 spoon17.ccs.ornl.gov kernel: [ 513.436152] Lustre: Unmounted lustre-client Sep 10 12:21:12 spoon17.ccs.ornl.gov kernel: [ 513.579799] Lustre: Mounted lustre-client Sep 10 12:21:12 spoon17.ccs.ornl.gov kernel: [ 513.621826] Lustre: *** cfs_fail_loc=1901, val=0*** Sep 10 12:21:12 spoon17.ccs.ornl.gov kernel: [ 514.053432] Lustre: Unmounted lustre-client Sep 10 12:21:12 spoon17.ccs.ornl.gov kernel: [ 514.402772] Lustre: DEBUG MARKER: == sanity test 300j: test large update record == 12:21:12 (1441902072) Sep 10 12:21:12 spoon17.ccs.ornl.gov kernel: [ 514.456732] Lustre: *** cfs_fail_loc=1702, val=0*** Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.846897] Lustre: DEBUG MARKER: == sanity test 300k: test large striped directory == 12:21:13 (1441902073) Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.901412] Lustre: *** cfs_fail_loc=1703, val=0*** Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.909345] Lustre: 30312:0:(osd_handler.c:1106:osd_trans_start()) lustre-MDT0000: too many transaction credits (51379 > 1024) Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.920902] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/0, destroy: 0/0/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.930904] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.941673] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.953641] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/0, delete: 0/0/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.963633] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.973621] Pid: 30312, comm: mdt01_005 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.977556] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.977557] Call Trace: Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.981711] [] libcfs_debug_dumpstack+0x55/0x80 [libcfs] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.988791] [] osd_trans_start+0x3ef/0x410 [osd_ldiskfs] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 514.995936] [] top_trans_start+0x357/0x960 [ptlrpc] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.002588] [] lod_trans_start+0x61/0x70 [lod] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.008800] [] mdd_trans_start+0x14/0x20 [mdd] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.015008] [] mdd_create+0x9cd/0x1280 [mdd] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.021056] [] ? mdt_version_save+0x8c/0x1a0 [mdt] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.027620] [] mdt_reint_create+0xbbc/0xcc0 [mdt] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.034122] [] ? lu_ucred+0x20/0x30 [obdclass] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.040386] [] ? mdt_ucred+0x15/0x20 [mdt] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.046284] [] ? mdt_root_squash+0x2c/0x3f0 [mdt] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.052812] [] ? __req_capsule_get+0x162/0x6e0 [ptlrpc] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.059811] [] mdt_reint_rec+0x5d/0x200 [mdt] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.065938] [] mdt_reint_internal+0x62b/0xb80 [mdt] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.072622] [] mdt_reint+0x6b/0x120 [mdt] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.078449] [] tgt_request_handle+0xa4c/0x1290 [ptlrpc] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.085555] [] ptlrpc_main+0xe41/0x1910 [ptlrpc] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.091995] [] ? ptlrpc_main+0x0/0x1910 [ptlrpc] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.098371] [] kthread+0x9e/0xc0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.103354] [] child_rip+0xa/0x20 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.108422] [] ? kthread+0x0/0xc0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.113536] [] ? child_rip+0x0/0x20 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.118808] Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.120489] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 3, rb = 4 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.423620] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/11, destroy: 0/0/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.433696] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 419 previous similar messages Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.443770] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.454535] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 419 previous similar messages Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.464662] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.475427] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 419 previous similar messages Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.485528] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/189, delete: 0/0/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.495730] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 419 previous similar messages Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.505805] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:21:13 spoon17.ccs.ornl.gov kernel: [ 515.515801] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 419 previous similar messages Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 515.622797] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 515.632881] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 472 previous similar messages Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.438443] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/14, destroy: 0/0/0 Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.448519] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 262 previous similar messages Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.458591] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.469357] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 262 previous similar messages Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.479429] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.490193] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 262 previous similar messages Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.500288] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/306, delete: 0/0/0 Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.510453] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 262 previous similar messages Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.520525] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:21:14 spoon17.ccs.ornl.gov kernel: [ 516.530520] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 262 previous similar messages Sep 10 12:21:15 spoon17.ccs.ornl.gov kernel: [ 516.635644] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:21:15 spoon17.ccs.ornl.gov kernel: [ 516.645710] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 228 previous similar messages Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.443684] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/15, destroy: 0/0/0 Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.453761] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 292 previous similar messages Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.463831] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.474597] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 292 previous similar messages Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.484664] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.495434] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 292 previous similar messages Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.505504] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/415, delete: 0/0/0 Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.515660] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 292 previous similar messages Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.525728] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:21:16 spoon17.ccs.ornl.gov kernel: [ 518.535708] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 292 previous similar messages Sep 10 12:21:17 spoon17.ccs.ornl.gov kernel: [ 518.660276] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:21:17 spoon17.ccs.ornl.gov kernel: [ 518.670342] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 289 previous similar messages Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.474142] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/20, destroy: 0/0/0 Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.484246] Lustre: 30312:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 423 previous similar messages Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.494320] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.505088] Lustre: 30312:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 423 previous similar messages Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.515227] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.525995] Lustre: 30312:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 423 previous similar messages Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.536067] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/540, delete: 0/0/0 Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.546338] Lustre: 30312:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 423 previous similar messages Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.556412] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:21:20 spoon17.ccs.ornl.gov kernel: [ 522.566397] Lustre: 30312:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 423 previous similar messages Sep 10 12:21:21 spoon17.ccs.ornl.gov kernel: [ 522.699596] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:21:21 spoon17.ccs.ornl.gov kernel: [ 522.709669] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 419 previous similar messages Sep 10 12:21:22 spoon17.ccs.ornl.gov kernel: [ 524.365258] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 7, rb = 7 Sep 10 12:21:22 spoon17.ccs.ornl.gov kernel: [ 524.457135] Lustre: 30306:0:(osd_handler.c:1106:osd_trans_start()) lustre-MDT0001: too many transaction credits (31382 > 1024) Sep 10 12:21:22 spoon17.ccs.ornl.gov kernel: [ 524.468692] Lustre: 30306:0:(osd_handler.c:1106:osd_trans_start()) Skipped 1 previous similar message Sep 10 12:21:25 spoon17.ccs.ornl.gov kernel: [ 526.644455] Lustre: 30312:0:(osd_handler.c:1106:osd_trans_start()) lustre-MDT0000: too many transaction credits (14144 > 1024) Sep 10 12:21:25 spoon17.ccs.ornl.gov kernel: [ 526.775302] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 7, rb = 7 Sep 10 12:21:25 spoon17.ccs.ornl.gov kernel: [ 526.785381] LustreError: 30312:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 35 previous similar messages Sep 10 12:21:25 spoon17.ccs.ornl.gov kernel: [ 527.136054] Lustre: DEBUG MARKER: == sanity test 300l: non-root user to create dir under striped dir with stale layout == 12:21:25 (1441902085) Sep 10 12:21:25 spoon17.ccs.ornl.gov kernel: [ 527.199405] Lustre: *** cfs_fail_loc=158, val=0*** Sep 10 12:21:25 spoon17.ccs.ornl.gov kernel: [ 527.204342] Lustre: Skipped 384 previous similar messages Sep 10 12:21:25 spoon17.ccs.ornl.gov kernel: [ 527.380184] Lustre: DEBUG MARKER: == sanity test complete, duration 87 sec == 12:21:25 (1441902085) Sep 10 12:32:24 spoon17.ccs.ornl.gov kernel: imklog 5.8.10, log source = /proc/kmsg started. Sep 10 12:33:14 spoon17.ccs.ornl.gov kernel: [ 159.117164] Lustre: 18906:0:(gss_svc_upcall.c:1129:gss_init_svc_upcall()) Init channel is not opened by lsvcgssd, following request might be dropped until lsvcgssd is active Sep 10 12:33:14 spoon17.ccs.ornl.gov kernel: [ 159.132775] Lustre: 18906:0:(gss_mech_switch.c:72:lgss_mech_register()) Register gssnull mechanism Sep 10 12:33:15 spoon17.ccs.ornl.gov kernel: [ 159.451019] Lustre: Echo OBD driver; http://www.lustre.org/ Sep 10 12:33:17 spoon17.ccs.ornl.gov kernel: [ 162.041397] Lustre: DEBUG MARKER: grep -c /tmp/mds1' ' /proc/mounts Sep 10 12:33:17 spoon17.ccs.ornl.gov kernel: [ 162.103193] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:33:18 spoon17.ccs.ornl.gov kernel: [ 162.357194] Lustre: DEBUG MARKER: mkfs.lustre --mgs --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reformat /tmp/lustre-mdt1 Sep 10 12:33:18 spoon17.ccs.ornl.gov kernel: [ 162.428965] zunicode: module license 'CDDL' taints kernel. Sep 10 12:33:18 spoon17.ccs.ornl.gov kernel: [ 162.434516] Disabling lock debugging due to kernel taint Sep 10 12:33:18 spoon17.ccs.ornl.gov kernel: [ 162.487941] SPL: Loaded module v0.6.4.2-1 Sep 10 12:33:18 spoon17.ccs.ornl.gov kernel: [ 162.744734] ZFS: Loaded module v0.6.4.2-1, ZFS pool version 5000, ZFS filesystem version 5 Sep 10 12:33:18 spoon17.ccs.ornl.gov kernel: [ 162.953290] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:19 spoon17.ccs.ornl.gov kernel: [ 163.401227] Lustre: DEBUG MARKER: grep -c /tmp/mds2' ' /proc/mounts Sep 10 12:33:19 spoon17.ccs.ornl.gov kernel: [ 163.466966] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:33:19 spoon17.ccs.ornl.gov kernel: [ 163.542627] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=1 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:33:19 spoon17.ccs.ornl.gov kernel: [ 163.760829] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:19 spoon17.ccs.ornl.gov kernel: [ 164.170668] Lustre: DEBUG MARKER: grep -c /tmp/mds3' ' /proc/mounts Sep 10 12:33:20 spoon17.ccs.ornl.gov kernel: [ 164.231168] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:33:20 spoon17.ccs.ornl.gov kernel: [ 164.305604] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=2 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:33:20 spoon17.ccs.ornl.gov kernel: [ 164.532689] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:20 spoon17.ccs.ornl.gov kernel: [ 165.036096] Lustre: DEBUG MARKER: grep -c /tmp/mds4' ' /proc/mounts Sep 10 12:33:20 spoon17.ccs.ornl.gov kernel: [ 165.104245] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:33:21 spoon17.ccs.ornl.gov kernel: [ 165.191788] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=3 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:33:21 spoon17.ccs.ornl.gov kernel: [ 165.451313] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:21 spoon17.ccs.ornl.gov kernel: [ 165.769106] Lustre: DEBUG MARKER: grep -c /tmp/ost1' ' /proc/mounts Sep 10 12:33:21 spoon17.ccs.ornl.gov kernel: [ 165.829783] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:33:21 spoon17.ccs.ornl.gov kernel: [ 165.905662] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --ost --index=0 --param=sys.timeout=20 --backfstype=ldiskfs --device-size=400000 --reformat /tmp/lustre-ost1 Sep 10 12:33:21 spoon17.ccs.ornl.gov kernel: [ 166.067360] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 166.384080] Lustre: DEBUG MARKER: grep -c /tmp/ost2' ' /proc/mounts Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 166.444190] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 166.518130] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --ost --index=1 --param=sys.timeout=20 --backfstype=ldiskfs --device-size=400000 --reformat /tmp/lustre-ost2 Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 166.685831] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 166.786529] Lustre: DEBUG MARKER: running=$(grep -c /tmp/ost1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/ost1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 166.872735] Lustre: DEBUG MARKER: running=$(grep -c /tmp/ost2' ' /proc/mounts); mpts=$(mount | grep -c /tmp/ost2' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 166.951093] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 167.029763] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds2' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds2' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:22 spoon17.ccs.ornl.gov kernel: [ 167.107947] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds3' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds3' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:23 spoon17.ccs.ornl.gov kernel: [ 167.187504] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds4' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds4' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:23 spoon17.ccs.ornl.gov kernel: [ 167.412491] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:23 spoon17.ccs.ornl.gov kernel: [ 167.452457] Lustre: DEBUG MARKER: running=$(grep -c /lustre/lustre' ' /proc/mounts); mpts=$(mount | grep -c /lustre/lustre' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:23 spoon17.ccs.ornl.gov kernel: [ 167.492186] Lustre: DEBUG MARKER: running=$(grep -c /lustre/testfs' ' /proc/mounts); mpts=$(mount | grep -c /lustre/testfs' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:33:24 spoon17.ccs.ornl.gov kernel: [ 168.948128] Lustre: DEBUG MARKER: mkdir -p /tmp/mds1 Sep 10 12:33:24 spoon17.ccs.ornl.gov kernel: [ 169.016600] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt1 Sep 10 12:33:24 spoon17.ccs.ornl.gov kernel: [ 169.087453] Lustre: DEBUG MARKER: mkdir -p /tmp/mds1; mount -t lustre -o loop /tmp/lustre-mdt1 /tmp/mds1 Sep 10 12:33:24 spoon17.ccs.ornl.gov kernel: [ 169.158215] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:24 spoon17.ccs.ornl.gov kernel: [ 169.170305] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.282321] Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.295816] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.350694] Lustre: lustre-MDT0000: new disk, initializing Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.369612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.502546] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.570348] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt1 2>/dev/null Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.633032] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.779749] Lustre: DEBUG MARKER: mkdir -p /tmp/mds2 Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.846317] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt2 Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.914373] Lustre: DEBUG MARKER: mkdir -p /tmp/mds2; mount -t lustre -o loop /tmp/lustre-mdt2 /tmp/mds2 Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.978817] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 169.998945] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 170.081443] Lustre: Setting parameter lustre-MDT0001-mdtlov.lov.stripesize in log lustre-MDT0001 Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 170.090456] Lustre: Skipped 4 previous similar messages Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 170.099306] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 170.106867] Lustre: Skipped 1 previous similar message Sep 10 12:33:25 spoon17.ccs.ornl.gov kernel: [ 170.160234] Lustre: lustre-MDT0001: new disk, initializing Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.187473] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400):1:mdt Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.198468] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400):1:mdt] Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.322449] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.391530] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt2 2>/dev/null Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.454421] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.603862] Lustre: DEBUG MARKER: mkdir -p /tmp/mds3 Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.669705] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt3 Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.736504] Lustre: DEBUG MARKER: mkdir -p /tmp/mds3; mount -t lustre -o loop /tmp/lustre-mdt3 /tmp/mds3 Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.800037] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.819524] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 170.986875] Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400):2:mdt] Sep 10 12:33:26 spoon17.ccs.ornl.gov kernel: [ 171.108715] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.177484] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt3 2>/dev/null Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.240154] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.387693] Lustre: DEBUG MARKER: mkdir -p /tmp/mds4 Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.454484] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt4 Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.521217] Lustre: DEBUG MARKER: mkdir -p /tmp/mds4; mount -t lustre -o loop /tmp/lustre-mdt4 /tmp/mds4 Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.587864] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.608885] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.695850] Lustre: Setting parameter lustre-MDT0003-mdtlov.lov.stripesize in log lustre-MDT0003 Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.704803] Lustre: Skipped 9 previous similar messages Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.713897] Lustre: srv-lustre-MDT0003: No data found on store. Initialize space Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.721503] Lustre: Skipped 1 previous similar message Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.777285] Lustre: lustre-MDT0003: new disk, initializing Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.782885] Lustre: Skipped 1 previous similar message Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.824509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400):3:mdt Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.835383] Lustre: Skipped 1 previous similar message Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 171.948060] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 172.015903] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt4 2>/dev/null Sep 10 12:33:27 spoon17.ccs.ornl.gov kernel: [ 172.078026] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.227161] Lustre: DEBUG MARKER: mkdir -p /tmp/ost1 Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.292984] Lustre: DEBUG MARKER: test -b /tmp/lustre-ost1 Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.360031] Lustre: DEBUG MARKER: mkdir -p /tmp/ost1; mount -t lustre -o loop /tmp/lustre-ost1 /tmp/ost1 Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.426908] LDISKFS-fs (loop4): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.454955] LDISKFS-fs (loop4): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.751817] Lustre: DEBUG MARKER: e2label /tmp/lustre-ost1 2>/dev/null Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.896915] Lustre: DEBUG MARKER: mkdir -p /tmp/ost2 Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 172.963060] Lustre: DEBUG MARKER: test -b /tmp/lustre-ost2 Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 173.030067] Lustre: DEBUG MARKER: mkdir -p /tmp/ost2; mount -t lustre -o loop /tmp/lustre-ost2 /tmp/ost2 Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 173.094808] LDISKFS-fs (loop5): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:28 spoon17.ccs.ornl.gov kernel: [ 173.123237] LDISKFS-fs (loop5): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:33:29 spoon17.ccs.ornl.gov kernel: [ 173.393788] Lustre: DEBUG MARKER: e2label /tmp/lustre-ost2 2>/dev/null Sep 10 12:33:29 spoon17.ccs.ornl.gov kernel: [ 173.458369] Lustre: DEBUG MARKER: mkdir -p /lustre/lustre Sep 10 12:33:29 spoon17.ccs.ornl.gov kernel: [ 173.477539] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.37.248.19@o2ib1:/lustre /lustre/lustre Sep 10 12:33:34 spoon17.ccs.ornl.gov kernel: [ 178.591680] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400):0:ost Sep 10 12:33:34 spoon17.ccs.ornl.gov kernel: [ 178.602587] Lustre: cli-ffff8801286ab044-su: Allocated super-sequence [0x0000000300000400-0x0000000340000400):0:ost] Sep 10 12:33:34 spoon17.ccs.ornl.gov kernel: [ 178.613314] Lustre: Skipped 2 previous similar messages Sep 10 12:33:39 spoon17.ccs.ornl.gov kernel: [ 183.731901] Lustre: Mounted lustre-client Sep 10 12:33:39 spoon17.ccs.ornl.gov kernel: [ 183.765793] Lustre: DEBUG MARKER: running=$(mount | grep -c /lustre/lustre' '); rc=0; if [ $running -eq 0 ] ; then mkdir -p /lustre/lustre; mount -t lustre -o user_xattr,flock 10.37.248.19@o2ib1:/lustre /lustre/lustre; rc=$?; fi; exit $rc Sep 10 12:33:39 spoon17.ccs.ornl.gov kernel: [ 183.803435] Lustre: DEBUG MARKER: mount | grep /lustre/lustre' ' Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 184.954266] Lustre: DEBUG MARKER: lctl get_param -n timeout Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 184.993444] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20 Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 185.008549] Lustre: DEBUG MARKER: Using TIMEOUT=20 Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 185.057810] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 185.083779] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 185.118588] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 185.141243] Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param lustre.sys.jobid_var='procname_uid' Sep 10 12:33:40 spoon17.ccs.ornl.gov kernel: [ 185.167079] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:33:41 spoon17.ccs.ornl.gov kernel: [ 185.188291] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:33:42 spoon17.ccs.ornl.gov kernel: [ 186.212309] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:33:43 spoon17.ccs.ornl.gov kernel: [ 187.236015] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:33:44 spoon17.ccs.ornl.gov kernel: [ 188.259770] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:34:09 spoon17.ccs.ornl.gov kernel: [ 214.086018] Lustre: DEBUG MARKER: Using TIMEOUT=20 Sep 10 12:34:10 spoon17.ccs.ornl.gov kernel: [ 214.518386] Lustre: DEBUG MARKER: excepting tests: 76 42a 42b 42c 42d 45 51d 68b Sep 10 12:34:10 spoon17.ccs.ornl.gov kernel: [ 214.555267] Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24D 27m 64b 68 71 77f 78 115 124b Sep 10 12:34:11 spoon17.ccs.ornl.gov kernel: [ 215.641588] Lustre: DEBUG MARKER: == sanity test 300a: basic striped dir sanity test == 12:34:11 (1441902851) Sep 10 12:34:12 spoon17.ccs.ornl.gov kernel: [ 216.194718] Lustre: DEBUG MARKER: == sanity test 300b: check ctime/mtime for striped dir == 12:34:11 (1441902851) Sep 10 12:34:32 spoon17.ccs.ornl.gov kernel: [ 236.771098] Lustre: DEBUG MARKER: == sanity test 300c: chown Sep 10 12:35:12 spoon17.ccs.ornl.gov kernel: [ 277.015420] Lustre: DEBUG MARKER: == sanity test 300d: check default stripe under striped directory == 12:35:12 (1441902912) Sep 10 12:35:13 spoon17.ccs.ornl.gov kernel: [ 277.861667] Lustre: DEBUG MARKER: == sanity test 300e: check rename under striped directory == 12:35:13 (1441902913) Sep 10 12:35:14 spoon17.ccs.ornl.gov kernel: [ 278.297393] Lustre: DEBUG MARKER: == sanity test 300f: check rename cross striped directory == 12:35:14 (1441902914) Sep 10 12:35:14 spoon17.ccs.ornl.gov kernel: [ 278.735235] Lustre: DEBUG MARKER: == sanity test 300g: check default striped directory for normal directory == 12:35:14 (1441902914) Sep 10 12:35:15 spoon17.ccs.ornl.gov kernel: [ 279.436168] Lustre: DEBUG MARKER: sanity test_300g: @@@@@@ FAIL: stripe count 1 != 0 for /lustre/lustre/d300g.sanity/normal_dir/test1 Sep 10 12:35:16 spoon17.ccs.ornl.gov kernel: [ 280.198073] Lustre: DEBUG MARKER: == sanity test 300h: check default striped directory for striped directory == 12:35:15 (1441902915) Sep 10 12:35:16 spoon17.ccs.ornl.gov kernel: [ 280.945135] Lustre: DEBUG MARKER: sanity test_300h: @@@@@@ FAIL: stripe count 1 != 0 for /lustre/lustre/d300h.sanity/striped_dir/test1 Sep 10 12:35:17 spoon17.ccs.ornl.gov kernel: [ 281.596837] Lustre: DEBUG MARKER: == sanity test 300i: client handle unknown hash type striped directory == 12:35:17 (1441902917) Sep 10 12:35:17 spoon17.ccs.ornl.gov kernel: [ 282.061020] Lustre: Unmounted lustre-client Sep 10 12:35:18 spoon17.ccs.ornl.gov kernel: [ 282.243038] Lustre: Mounted lustre-client Sep 10 12:35:18 spoon17.ccs.ornl.gov kernel: [ 282.283080] Lustre: *** cfs_fail_loc=1901, val=0*** Sep 10 12:35:18 spoon17.ccs.ornl.gov kernel: [ 282.719099] Lustre: Unmounted lustre-client Sep 10 12:35:18 spoon17.ccs.ornl.gov kernel: [ 283.073555] Lustre: DEBUG MARKER: == sanity test 300j: test large update record == 12:35:18 (1441902918) Sep 10 12:35:18 spoon17.ccs.ornl.gov kernel: [ 283.126569] Lustre: *** cfs_fail_loc=1702, val=0*** Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.502856] Lustre: DEBUG MARKER: == sanity test 300k: test large striped directory == 12:35:19 (1441902919) Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.555631] Lustre: *** cfs_fail_loc=1703, val=0*** Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.563480] Lustre: 29844:0:(osd_handler.c:1106:osd_trans_start()) lustre-MDT0000: too many transaction credits (51379 > 1024) Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.575080] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/0, destroy: 0/0/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.585090] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.597106] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.607934] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/0, delete: 0/0/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.617951] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.627962] Pid: 29844, comm: mdt01_005 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.631922] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.631923] Call Trace: Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.636098] [] libcfs_debug_dumpstack+0x55/0x80 [libcfs] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.643201] [] osd_trans_start+0x3ef/0x410 [osd_ldiskfs] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.650441] [] top_trans_start+0x357/0x960 [ptlrpc] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.657134] [] lod_trans_start+0x61/0x70 [lod] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.663447] [] mdd_trans_start+0x14/0x20 [mdd] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.669699] [] mdd_create+0x9cd/0x1280 [mdd] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.675797] [] ? mdt_version_save+0x8c/0x1a0 [mdt] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.682471] [] mdt_reint_create+0xbbc/0xcc0 [mdt] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.688988] [] ? lu_ucred+0x20/0x30 [obdclass] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.695246] [] ? mdt_ucred+0x15/0x20 [mdt] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.701251] [] ? mdt_root_squash+0x2c/0x3f0 [mdt] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.707889] [] ? __req_capsule_get+0x162/0x6e0 [ptlrpc] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.714961] [] mdt_reint_rec+0x5d/0x200 [mdt] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.721162] [] mdt_reint_internal+0x62b/0xb80 [mdt] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.727856] [] mdt_reint+0x6b/0x120 [mdt] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.733745] [] tgt_request_handle+0xa4c/0x1290 [ptlrpc] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.740842] [] ptlrpc_main+0xe41/0x1910 [ptlrpc] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.747326] [] ? ptlrpc_main+0x0/0x1910 [ptlrpc] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.753823] [] kthread+0x9e/0xc0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.758849] [] child_rip+0xa/0x20 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.763991] [] ? kthread+0x0/0xc0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.769101] [] ? child_rip+0x0/0x20 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.774450] Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 283.776164] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 3, rb = 4 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.077327] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/11, destroy: 0/0/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.087401] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 418 previous similar messages Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.097511] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.108293] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 418 previous similar messages Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.118406] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.129243] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 418 previous similar messages Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.139349] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/189, delete: 0/0/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.149523] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 418 previous similar messages Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.159655] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:35:19 spoon17.ccs.ornl.gov kernel: [ 284.169655] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 418 previous similar messages Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 284.282637] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 284.292784] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 460 previous similar messages Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.093328] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/16, destroy: 0/0/0 Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.103445] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 223 previous similar messages Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.113553] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.124398] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 223 previous similar messages Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.134498] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.145295] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 223 previous similar messages Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.155390] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/286, delete: 0/0/0 Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.165588] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 223 previous similar messages Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.175829] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:35:20 spoon17.ccs.ornl.gov kernel: [ 285.185862] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 223 previous similar messages Sep 10 12:35:21 spoon17.ccs.ornl.gov kernel: [ 285.287970] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 2, rb = 2 Sep 10 12:35:21 spoon17.ccs.ornl.gov kernel: [ 285.298076] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 197 previous similar messages Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.099754] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/18, destroy: 0/0/0 Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.109833] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 258 previous similar messages Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.119944] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.130805] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 258 previous similar messages Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.140974] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.151828] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 258 previous similar messages Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.162026] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/391, delete: 0/0/0 Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.172297] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 258 previous similar messages Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.182411] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:35:22 spoon17.ccs.ornl.gov kernel: [ 287.192457] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 258 previous similar messages Sep 10 12:35:23 spoon17.ccs.ornl.gov kernel: [ 287.300611] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:35:23 spoon17.ccs.ornl.gov kernel: [ 287.310704] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 254 previous similar messages Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.126707] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/19, destroy: 0/0/0 Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.136785] Lustre: 29844:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 406 previous similar messages Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.146862] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.157690] Lustre: 29844:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 406 previous similar messages Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.167765] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.178530] Lustre: 29844:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 406 previous similar messages Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.188596] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/513, delete: 0/0/0 Sep 10 12:35:26 spoon17.ccs.ornl.gov kernel: [ 291.198755] Lustre: 29844:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 406 previous similar messages Sep 10 12:35:27 spoon17.ccs.ornl.gov kernel: [ 291.208831] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:35:27 spoon17.ccs.ornl.gov kernel: [ 291.218814] Lustre: 29844:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 406 previous similar messages Sep 10 12:35:27 spoon17.ccs.ornl.gov kernel: [ 291.302337] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:35:27 spoon17.ccs.ornl.gov kernel: [ 291.312406] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 404 previous similar messages Sep 10 12:35:29 spoon17.ccs.ornl.gov kernel: [ 293.835247] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 7, rb = 7 Sep 10 12:35:29 spoon17.ccs.ornl.gov kernel: [ 293.898973] Lustre: 29725:0:(osd_handler.c:1106:osd_trans_start()) lustre-MDT0001: too many transaction credits (31382 > 1024) Sep 10 12:35:29 spoon17.ccs.ornl.gov kernel: [ 293.910524] Lustre: 29725:0:(osd_handler.c:1106:osd_trans_start()) Skipped 2 previous similar messages Sep 10 12:35:31 spoon17.ccs.ornl.gov kernel: [ 295.962930] Lustre: 29844:0:(osd_handler.c:1106:osd_trans_start()) lustre-MDT0000: too many transaction credits (14144 > 1024) Sep 10 12:35:31 spoon17.ccs.ornl.gov kernel: [ 296.093807] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 7, rb = 7 Sep 10 12:35:31 spoon17.ccs.ornl.gov kernel: [ 296.103885] LustreError: 29844:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 35 previous similar messages Sep 10 12:35:32 spoon17.ccs.ornl.gov kernel: [ 296.475976] Lustre: DEBUG MARKER: == sanity test 300l: non-root user to create dir under striped dir with stale layout == 12:35:32 (1441902932) Sep 10 12:35:32 spoon17.ccs.ornl.gov kernel: [ 296.541151] Lustre: *** cfs_fail_loc=158, val=0*** Sep 10 12:35:32 spoon17.ccs.ornl.gov kernel: [ 296.546072] Lustre: Skipped 384 previous similar messages Sep 10 12:35:32 spoon17.ccs.ornl.gov kernel: [ 296.746735] Lustre: DEBUG MARKER: == sanity test complete, duration 87 sec == 12:35:32 (1441902932) Sep 10 12:35:48 spoon17.ccs.ornl.gov kernel: [ 312.628815] ACPI: No handler for Region [IPMI] (ffff8801ad4c7420) [IPMI] Sep 10 12:35:51 spoon17.ccs.ornl.gov kernel: [ 316.067509] ACPI: No handler for Region [IPMI] (ffff8801ad4c7420) [IPMI] Sep 10 12:36:26 spoon17.ccs.ornl.gov kernel: [ 350.470693] ACPI: No handler for Region [IPMI] (ffff8801ad4c7420) [IPMI] Sep 10 12:47:37 spoon17.ccs.ornl.gov kernel: imklog 5.8.10, log source = /proc/kmsg started. Sep 10 12:47:57 spoon17.ccs.ornl.gov kernel: [ 133.347143] Lustre: 19974:0:(gss_svc_upcall.c:1129:gss_init_svc_upcall()) Init channel is not opened by lsvcgssd, following request might be dropped until lsvcgssd is active Sep 10 12:47:57 spoon17.ccs.ornl.gov kernel: [ 133.362679] Lustre: 19974:0:(gss_mech_switch.c:72:lgss_mech_register()) Register gssnull mechanism Sep 10 12:47:57 spoon17.ccs.ornl.gov kernel: [ 133.663030] Lustre: Echo OBD driver; http://www.lustre.org/ Sep 10 12:48:00 spoon17.ccs.ornl.gov kernel: [ 135.994046] Lustre: DEBUG MARKER: grep -c /tmp/mds1' ' /proc/mounts Sep 10 12:48:00 spoon17.ccs.ornl.gov kernel: [ 136.056219] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:48:00 spoon17.ccs.ornl.gov kernel: [ 136.318645] Lustre: DEBUG MARKER: mkfs.lustre --mgs --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reformat /tmp/lustre-mdt1 Sep 10 12:48:00 spoon17.ccs.ornl.gov kernel: [ 136.392150] zunicode: module license 'CDDL' taints kernel. Sep 10 12:48:00 spoon17.ccs.ornl.gov kernel: [ 136.397729] Disabling lock debugging due to kernel taint Sep 10 12:48:00 spoon17.ccs.ornl.gov kernel: [ 136.450666] SPL: Loaded module v0.6.4.2-1 Sep 10 12:48:00 spoon17.ccs.ornl.gov kernel: [ 136.704066] ZFS: Loaded module v0.6.4.2-1, ZFS pool version 5000, ZFS filesystem version 5 Sep 10 12:48:01 spoon17.ccs.ornl.gov kernel: [ 136.885863] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:01 spoon17.ccs.ornl.gov kernel: [ 137.224318] Lustre: DEBUG MARKER: grep -c /tmp/mds2' ' /proc/mounts Sep 10 12:48:01 spoon17.ccs.ornl.gov kernel: [ 137.273427] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:48:01 spoon17.ccs.ornl.gov kernel: [ 137.335666] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=1 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:48:01 spoon17.ccs.ornl.gov kernel: [ 137.506795] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:01 spoon17.ccs.ornl.gov kernel: [ 137.850812] Lustre: DEBUG MARKER: grep -c /tmp/mds3' ' /proc/mounts Sep 10 12:48:02 spoon17.ccs.ornl.gov kernel: [ 137.911773] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:48:02 spoon17.ccs.ornl.gov kernel: [ 137.987343] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=2 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:48:02 spoon17.ccs.ornl.gov kernel: [ 138.208929] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:02 spoon17.ccs.ornl.gov kernel: [ 138.636602] Lustre: DEBUG MARKER: grep -c /tmp/mds4' ' /proc/mounts Sep 10 12:48:02 spoon17.ccs.ornl.gov kernel: [ 138.697705] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:48:02 spoon17.ccs.ornl.gov kernel: [ 138.773104] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --mdt --index=3 --param=sys.timeout=20 --param=lov.stripesize=1048576 --param=lov.stripecount=0 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=200000 --reform Sep 10 12:48:03 spoon17.ccs.ornl.gov kernel: [ 138.994145] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:03 spoon17.ccs.ornl.gov kernel: [ 139.319396] Lustre: DEBUG MARKER: grep -c /tmp/ost1' ' /proc/mounts Sep 10 12:48:03 spoon17.ccs.ornl.gov kernel: [ 139.379803] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:48:03 spoon17.ccs.ornl.gov kernel: [ 139.454439] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --ost --index=0 --param=sys.timeout=20 --backfstype=ldiskfs --device-size=400000 --reformat /tmp/lustre-ost1 Sep 10 12:48:03 spoon17.ccs.ornl.gov kernel: [ 139.617121] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 139.940951] Lustre: DEBUG MARKER: grep -c /tmp/ost2' ' /proc/mounts Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.003085] Lustre: DEBUG MARKER: lsmod | grep lnet > /dev/null && lctl dl | grep ' ST ' Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.078138] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.248.19@o2ib1 --fsname=lustre --ost --index=1 --param=sys.timeout=20 --backfstype=ldiskfs --device-size=400000 --reformat /tmp/lustre-ost2 Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.243188] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.343875] Lustre: DEBUG MARKER: running=$(grep -c /tmp/ost1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/ost1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.425435] Lustre: DEBUG MARKER: running=$(grep -c /tmp/ost2' ' /proc/mounts); mpts=$(mount | grep -c /tmp/ost2' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.505667] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.586774] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds2' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds2' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.667743] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds3' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds3' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:04 spoon17.ccs.ornl.gov kernel: [ 140.747836] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds4' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds4' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:05 spoon17.ccs.ornl.gov kernel: [ 140.976729] Lustre: DEBUG MARKER: running=$(grep -c /tmp/mds1' ' /proc/mounts); mpts=$(mount | grep -c /tmp/mds1' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:05 spoon17.ccs.ornl.gov kernel: [ 141.016814] Lustre: DEBUG MARKER: running=$(grep -c /lustre/lustre' ' /proc/mounts); mpts=$(mount | grep -c /lustre/lustre' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:05 spoon17.ccs.ornl.gov kernel: [ 141.054642] Lustre: DEBUG MARKER: running=$(grep -c /lustre/testfs' ' /proc/mounts); mpts=$(mount | grep -c /lustre/testfs' '); if [ $running -ne $mpts ]; then echo $(hostname) env are INSANE!; exit 1; fi Sep 10 12:48:06 spoon17.ccs.ornl.gov kernel: [ 142.553278] Lustre: DEBUG MARKER: mkdir -p /tmp/mds1 Sep 10 12:48:06 spoon17.ccs.ornl.gov kernel: [ 142.622533] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt1 Sep 10 12:48:06 spoon17.ccs.ornl.gov kernel: [ 142.693923] Lustre: DEBUG MARKER: mkdir -p /tmp/mds1; mount -t lustre -o loop /tmp/lustre-mdt1 /tmp/mds1 Sep 10 12:48:06 spoon17.ccs.ornl.gov kernel: [ 142.764521] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:06 spoon17.ccs.ornl.gov kernel: [ 142.777187] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 142.900549] Lustre: Setting parameter lustre-MDT0000-mdtlov.lov.stripesize in log lustre-MDT0000 Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 142.914562] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 142.969782] Lustre: lustre-MDT0000: new disk, initializing Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 142.989811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400):0:mdt Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.128518] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.199717] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt1 2>/dev/null Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.265806] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.418895] Lustre: DEBUG MARKER: mkdir -p /tmp/mds2 Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.488881] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt2 Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.558679] Lustre: DEBUG MARKER: mkdir -p /tmp/mds2; mount -t lustre -o loop /tmp/lustre-mdt2 /tmp/mds2 Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.627516] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.647591] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.730227] Lustre: Setting parameter lustre-MDT0001-mdtlov.lov.stripesize in log lustre-MDT0001 Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.739201] Lustre: Skipped 4 previous similar messages Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.748689] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.756245] Lustre: Skipped 1 previous similar message Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.811678] Lustre: lustre-MDT0001: new disk, initializing Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.840303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400):1:mdt Sep 10 12:48:07 spoon17.ccs.ornl.gov kernel: [ 143.851400] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400):1:mdt] Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 143.970062] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.039215] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt2 2>/dev/null Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.103592] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.257448] Lustre: DEBUG MARKER: mkdir -p /tmp/mds3 Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.326810] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt3 Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.395839] Lustre: DEBUG MARKER: mkdir -p /tmp/mds3; mount -t lustre -o loop /tmp/lustre-mdt3 /tmp/mds3 Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.462629] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.490633] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.658720] Lustre: cli-ctl-lustre-MDT0002: Allocated super-sequence [0x0000000280000400-0x00000002c0000400):2:mdt] Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.801036] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:08 spoon17.ccs.ornl.gov kernel: [ 144.872217] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt3 2>/dev/null Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 144.937035] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.091160] Lustre: DEBUG MARKER: mkdir -p /tmp/mds4 Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.159068] Lustre: DEBUG MARKER: test -b /tmp/lustre-mdt4 Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.227884] Lustre: DEBUG MARKER: mkdir -p /tmp/mds4; mount -t lustre -o loop /tmp/lustre-mdt4 /tmp/mds4 Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.294619] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.315408] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.402742] Lustre: Setting parameter lustre-MDT0003-mdtlov.lov.stripesize in log lustre-MDT0003 Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.411724] Lustre: Skipped 9 previous similar messages Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.420758] Lustre: srv-lustre-MDT0003: No data found on store. Initialize space Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.428364] Lustre: Skipped 1 previous similar message Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.483440] Lustre: lustre-MDT0003: new disk, initializing Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.489038] Lustre: Skipped 1 previous similar message Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.530706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400):3:mdt Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.541589] Lustre: Skipped 1 previous similar message Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.658186] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.729315] Lustre: DEBUG MARKER: e2label /tmp/lustre-mdt4 2>/dev/null Sep 10 12:48:09 spoon17.ccs.ornl.gov kernel: [ 145.794291] Lustre: DEBUG MARKER: lctl set_param -n mdt.lustre*.enable_remote_dir=1 Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 145.958380] Lustre: DEBUG MARKER: mkdir -p /tmp/ost1 Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.026928] Lustre: DEBUG MARKER: test -b /tmp/lustre-ost1 Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.097044] Lustre: DEBUG MARKER: mkdir -p /tmp/ost1; mount -t lustre -o loop /tmp/lustre-ost1 /tmp/ost1 Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.167258] LDISKFS-fs (loop4): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.186603] LDISKFS-fs (loop4): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.488985] Lustre: DEBUG MARKER: e2label /tmp/lustre-ost1 2>/dev/null Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.646474] Lustre: DEBUG MARKER: mkdir -p /tmp/ost2 Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.712788] Lustre: DEBUG MARKER: test -b /tmp/lustre-ost2 Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.780057] Lustre: DEBUG MARKER: mkdir -p /tmp/ost2; mount -t lustre -o loop /tmp/lustre-ost2 /tmp/ost2 Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.849264] LDISKFS-fs (loop5): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:10 spoon17.ccs.ornl.gov kernel: [ 146.877718] LDISKFS-fs (loop5): mounted filesystem with ordered data mode. quota=on. Opts: Sep 10 12:48:11 spoon17.ccs.ornl.gov kernel: [ 147.166114] Lustre: DEBUG MARKER: e2label /tmp/lustre-ost2 2>/dev/null Sep 10 12:48:11 spoon17.ccs.ornl.gov kernel: [ 147.233373] Lustre: DEBUG MARKER: mkdir -p /lustre/lustre Sep 10 12:48:11 spoon17.ccs.ornl.gov kernel: [ 147.253369] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock 10.37.248.19@o2ib1:/lustre /lustre/lustre Sep 10 12:48:12 spoon17.ccs.ornl.gov kernel: [ 148.780870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000300000400-0x0000000340000400):0:ost Sep 10 12:48:12 spoon17.ccs.ornl.gov kernel: [ 148.791775] Lustre: cli-ffff880129af6044-su: Allocated super-sequence [0x0000000300000400-0x0000000340000400):0:ost] Sep 10 12:48:12 spoon17.ccs.ornl.gov kernel: [ 148.802475] Lustre: Skipped 2 previous similar messages Sep 10 12:48:18 spoon17.ccs.ornl.gov kernel: [ 153.920233] Lustre: Mounted lustre-client Sep 10 12:48:18 spoon17.ccs.ornl.gov kernel: [ 153.955180] Lustre: DEBUG MARKER: running=$(mount | grep -c /lustre/lustre' '); rc=0; if [ $running -eq 0 ] ; then mkdir -p /lustre/lustre; mount -t lustre -o user_xattr,flock 10.37.248.19@o2ib1:/lustre /lustre/lustre; rc=$?; fi; exit $rc Sep 10 12:48:18 spoon17.ccs.ornl.gov kernel: [ 153.993065] Lustre: DEBUG MARKER: mount | grep /lustre/lustre' ' Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.148460] Lustre: DEBUG MARKER: lctl get_param -n timeout Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.188884] Lustre: DEBUG MARKER: /usr/sbin/lctl mark Using TIMEOUT=20 Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.203871] Lustre: DEBUG MARKER: Using TIMEOUT=20 Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.255422] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.281582] Lustre: DEBUG MARKER: lctl dl | grep ' IN osc ' 2>/dev/null | wc -l Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.315560] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.338276] Lustre: DEBUG MARKER: /usr/sbin/lctl conf_param lustre.sys.jobid_var='procname_uid' Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.365280] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:48:19 spoon17.ccs.ornl.gov kernel: [ 155.385832] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:48:20 spoon17.ccs.ornl.gov kernel: [ 156.409266] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:48:21 spoon17.ccs.ornl.gov kernel: [ 157.433683] Lustre: DEBUG MARKER: /usr/sbin/lctl get_param -n jobid_var Sep 10 12:48:39 spoon17.ccs.ornl.gov kernel: [ 175.623293] Lustre: DEBUG MARKER: Using TIMEOUT=20 Sep 10 12:48:40 spoon17.ccs.ornl.gov kernel: [ 176.058414] Lustre: DEBUG MARKER: excepting tests: 76 42a 42b 42c 42d 45 51d 68b Sep 10 12:48:40 spoon17.ccs.ornl.gov kernel: [ 176.096380] Lustre: DEBUG MARKER: skipping tests SLOW=no: 24o 24D 27m 64b 68 71 77f 78 115 124b Sep 10 12:48:41 spoon17.ccs.ornl.gov kernel: [ 177.171796] Lustre: DEBUG MARKER: == sanity test 300a: basic striped dir sanity test == 12:48:41 (1441903721) Sep 10 12:48:41 spoon17.ccs.ornl.gov kernel: [ 177.727341] Lustre: DEBUG MARKER: == sanity test 300b: check ctime/mtime for striped dir == 12:48:41 (1441903721) Sep 10 12:49:02 spoon17.ccs.ornl.gov kernel: [ 198.371025] Lustre: DEBUG MARKER: == sanity test 300c: chown Sep 10 12:49:41 spoon17.ccs.ornl.gov kernel: [ 237.855419] Lustre: DEBUG MARKER: == sanity test 300d: check default stripe under striped directory == 12:49:41 (1441903781) Sep 10 12:49:42 spoon17.ccs.ornl.gov kernel: [ 238.695929] Lustre: DEBUG MARKER: == sanity test 300e: check rename under striped directory == 12:49:42 (1441903782) Sep 10 12:49:43 spoon17.ccs.ornl.gov kernel: [ 239.133854] Lustre: DEBUG MARKER: == sanity test 300f: check rename cross striped directory == 12:49:43 (1441903783) Sep 10 12:49:43 spoon17.ccs.ornl.gov kernel: [ 239.577103] Lustre: DEBUG MARKER: == sanity test 300g: check default striped directory for normal directory == 12:49:43 (1441903783) Sep 10 12:49:44 spoon17.ccs.ornl.gov kernel: [ 240.291185] Lustre: DEBUG MARKER: sanity test_300g: @@@@@@ FAIL: stripe count 1 != 0 for /lustre/lustre/d300g.sanity/normal_dir/test1 Sep 10 12:49:45 spoon17.ccs.ornl.gov kernel: [ 241.048817] Lustre: DEBUG MARKER: == sanity test 300h: check default striped directory for striped directory == 12:49:45 (1441903785) Sep 10 12:49:45 spoon17.ccs.ornl.gov kernel: [ 241.837071] Lustre: DEBUG MARKER: sanity test_300h: @@@@@@ FAIL: stripe count 1 != 0 for /lustre/lustre/d300h.sanity/striped_dir/test1 Sep 10 12:49:46 spoon17.ccs.ornl.gov kernel: [ 242.491478] Lustre: DEBUG MARKER: == sanity test 300i: client handle unknown hash type striped directory == 12:49:46 (1441903786) Sep 10 12:49:47 spoon17.ccs.ornl.gov kernel: [ 242.967447] Lustre: Unmounted lustre-client Sep 10 12:49:47 spoon17.ccs.ornl.gov kernel: [ 243.114485] Lustre: Mounted lustre-client Sep 10 12:49:47 spoon17.ccs.ornl.gov kernel: [ 243.154990] Lustre: *** cfs_fail_loc=1901, val=0*** Sep 10 12:49:47 spoon17.ccs.ornl.gov kernel: [ 243.558132] Lustre: Unmounted lustre-client Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 243.918056] Lustre: DEBUG MARKER: == sanity test 300j: test large update record == 12:49:47 (1441903787) Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 243.972167] Lustre: *** cfs_fail_loc=1702, val=0*** Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.343613] Lustre: DEBUG MARKER: == sanity test 300k: test large striped directory == 12:49:48 (1441903788) Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.396698] Lustre: *** cfs_fail_loc=1703, val=0*** Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.404667] Lustre: 30319:0:(osd_handler.c:1106:osd_trans_start()) lustre-MDT0000: too many transaction credits (51379 > 1024) Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.416377] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/0, destroy: 0/0/0 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.426458] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.438443] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.449283] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/0, delete: 0/0/0 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.459406] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.469441] Pid: 30319, comm: mdt01_005 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.473450] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.473451] Call Trace: Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.477674] [] libcfs_debug_dumpstack+0x55/0x80 [libcfs] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.484822] [] osd_trans_start+0x3ef/0x410 [osd_ldiskfs] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.492041] [] top_trans_start+0x357/0x960 [ptlrpc] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.498758] [] lod_trans_start+0x61/0x70 [lod] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.505040] [] mdd_trans_start+0x14/0x20 [mdd] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.511315] [] mdd_create+0x9cd/0x1280 [mdd] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.517458] [] ? mdt_version_save+0x8c/0x1a0 [mdt] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.524033] [] mdt_reint_create+0xbbc/0xcc0 [mdt] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.530617] [] ? lu_ucred+0x20/0x30 [obdclass] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.536904] [] ? mdt_ucred+0x15/0x20 [mdt] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.542844] [] ? mdt_root_squash+0x2c/0x3f0 [mdt] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.549480] [] ? __req_capsule_get+0x162/0x6e0 [ptlrpc] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.556556] [] mdt_reint_rec+0x5d/0x200 [mdt] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.562756] [] mdt_reint_internal+0x62b/0xb80 [mdt] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.569480] [] mdt_reint+0x6b/0x120 [mdt] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.575392] [] tgt_request_handle+0xa4c/0x1290 [ptlrpc] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.582526] [] ptlrpc_main+0xe41/0x1910 [ptlrpc] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.589036] [] ? ptlrpc_main+0x0/0x1910 [ptlrpc] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.595487] [] kthread+0x9e/0xc0 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.600545] [] child_rip+0xa/0x20 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.605688] [] ? kthread+0x0/0xc0 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.610822] [] ? child_rip+0x0/0x20 Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.616129] Sep 10 12:49:48 spoon17.ccs.ornl.gov kernel: [ 244.617942] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 3, rb = 4 Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.919990] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/10, destroy: 0/0/0 Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.930062] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.940137] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.950905] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.960978] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.971748] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.981819] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/185, delete: 0/0/0 Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 244.991980] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 245.002049] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 245.012041] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 245.120021] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 2, rb = 2 Sep 10 12:49:49 spoon17.ccs.ornl.gov kernel: [ 245.130094] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 457 previous similar messages Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.925895] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/16, destroy: 0/0/0 Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.935963] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 278 previous similar messages Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.946043] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.956808] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 278 previous similar messages Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.966909] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.977675] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 278 previous similar messages Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.987779] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/309, delete: 0/0/0 Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 245.997982] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 278 previous similar messages Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 246.008061] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 246.018162] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 278 previous similar messages Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 246.125713] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:49:50 spoon17.ccs.ornl.gov kernel: [ 246.135782] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 252 previous similar messages Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 247.948049] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/18, destroy: 0/0/0 Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 247.958165] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 301 previous similar messages Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 247.968242] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 247.979018] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 301 previous similar messages Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 247.989162] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 247.999965] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 301 previous similar messages Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 248.010033] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/418, delete: 0/0/0 Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 248.020186] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 301 previous similar messages Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 248.030252] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 248.040229] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 301 previous similar messages Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 248.147262] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:49:52 spoon17.ccs.ornl.gov kernel: [ 248.157329] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 296 previous similar messages Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 251.955204] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) create: 146/1168/22, destroy: 0/0/0 Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 251.965281] Lustre: 30319:0:(osd_handler.c:1041:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 251.975351] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) attr_set: 130/130/0, xattr_set: 259/16513/0 Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 251.986115] Lustre: 30319:0:(osd_handler.c:1048:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 251.996192] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) write: 70/2146/0, punch: 0/0/0, quota 2/2/0 Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 252.006965] Lustre: 30319:0:(osd_handler.c:1058:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 252.017035] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) insert: 932/30778/539, delete: 0/0/0 Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 252.027204] Lustre: 30319:0:(osd_handler.c:1065:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 252.037277] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) ref_add: 642/642/0, ref_del: 0/0/0 Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 252.047258] Lustre: 30319:0:(osd_handler.c:1072:osd_trans_dump_creds()) Skipped 412 previous similar messages Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 252.211374] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 8, rb = 9 Sep 10 12:49:56 spoon17.ccs.ornl.gov kernel: [ 252.221446] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) Skipped 402 previous similar messages Sep 10 12:49:57 spoon17.ccs.ornl.gov kernel: [ 253.609802] LustreError: 30319:0:(osd_internal.h:1030:osd_trans_exec_op()) lustre-MDT0000-osd: op = 7, rb = 7 Sep 10 12:53:07 spoon17.ccs.ornl.gov kernel: imklog 5.8.10, log source = /proc/kmsg started.