Oct 11 12:54:14 maid-mds2 kernel: Lustre: Lustre OSC module (ffffffff88b75ae0). Oct 11 12:54:14 maid-mds2 kernel: Lustre: Lustre LOV module (ffffffff88be3520). Oct 11 12:54:15 maid-mds2 kernel: Lustre: Lustre client module (ffffffff88c83d00). Oct 11 12:54:34 maid-mds2 kernel: LDISKFS-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended Oct 11 12:54:34 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:54:34 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:54:34 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:54:34 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:54:34 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:54:34 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:54:34 maid-mds2 kernel: Lustre: debug daemon will attempt to start writing to /tmp/perflog.20885 (10485760kB max) Oct 11 12:54:34 maid-mds2 kernel: Lustre: shutting down debug daemon thread... Oct 11 12:56:43 maid-mds2 kernel: Lustre: Echo OBD driver; http://www.lustre.org/ Oct 11 12:56:43 maid-mds2 kernel: Lustre: Lustre OSC module (ffffffff88b75ae0). Oct 11 12:56:43 maid-mds2 kernel: Lustre: Lustre LOV module (ffffffff88be3520). Oct 11 12:56:43 maid-mds2 kernel: Lustre: Lustre client module (ffffffff88c83d00). Oct 11 12:57:02 maid-mds2 kernel: LDISKFS-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended Oct 11 12:57:02 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:57:02 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:57:02 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:57:02 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:57:02 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:57:03 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:57:03 maid-mds2 kernel: Lustre: debug daemon will attempt to start writing to /tmp/perflog.21380 (10485760kB max) Oct 11 12:57:03 maid-mds2 kernel: Lustre: shutting down debug daemon thread... Oct 11 12:57:04 maid-mds2 kernel: Lustre: Failing over lustre-MDT0000 Oct 11 12:57:04 maid-mds2 kernel: Lustre: Skipped 11 previous similar messages Oct 11 12:58:33 maid-mds2 kernel: Lustre: Echo OBD driver; http://www.lustre.org/ Oct 11 12:58:33 maid-mds2 kernel: Lustre: Lustre OSC module (ffffffff88b75ae0). Oct 11 12:58:33 maid-mds2 kernel: Lustre: Lustre LOV module (ffffffff88be3520). Oct 11 12:58:33 maid-mds2 kernel: Lustre: Lustre client module (ffffffff88c83d00). Oct 11 12:58:52 maid-mds2 kernel: LDISKFS-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended Oct 11 12:58:52 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:58:52 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:58:52 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:58:52 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:58:52 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 12:58:52 maid-mds2 kernel: Lustre: MGS MGS started Oct 11 12:58:52 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:58:52 maid-mds2 kernel: Lustre: import MGC10.37.248.100@o2ib1->MGC10.37.248.100@o2ib1_0 netid 90000: select flavor null Oct 11 12:58:52 maid-mds2 kernel: Lustre: Skipped 5 previous similar messages Oct 11 12:58:52 maid-mds2 kernel: Lustre: 21983:0:(ldlm_lib.c:902:target_handle_connect()) MGS: connection from ccafb608-4c7a-152e-d9bb-d539dd9bbb30@0@lo t0 exp 0000000000000000 cur 1318352332 last 0 Oct 11 12:58:52 maid-mds2 kernel: Lustre: 21983:0:(ldlm_lib.c:902:target_handle_connect()) Skipped 2 previous similar messages Oct 11 12:58:52 maid-mds2 kernel: Lustre: MGC10.37.248.100@o2ib1: Reactivating import Oct 11 12:58:52 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:58:52 maid-mds2 kernel: Lustre: MGS: Regenerating lustre-MDTffff log by user request. Oct 11 12:58:52 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:58:52 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 12:58:52 maid-mds2 kernel: Lustre: Enabling ACL Oct 11 12:58:52 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:58:52 maid-mds2 kernel: Lustre: Enabling user_xattr Oct 11 12:58:52 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:58:52 maid-mds2 kernel: Lustre: lustre-MDT0000: new disk, initializing Oct 11 12:58:52 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:59:47 maid-mds2 kernel: Lustre: 22044:0:(quota_master.c:793:close_quota_files()) quota[0] is off already Oct 11 12:59:47 maid-mds2 kernel: Lustre: 22044:0:(quota_master.c:793:close_quota_files()) Skipped 5 previous similar messages Oct 11 12:59:47 maid-mds2 kernel: Lustre: mdd_obd-lustre-MDT0000: shutting down for failover; client state will be preserved. Oct 11 12:59:47 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:59:47 maid-mds2 kernel: LustreError: 22044:0:(ldlm_request.c:1172:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway Oct 11 12:59:47 maid-mds2 kernel: LustreError: 22044:0:(ldlm_request.c:1172:ldlm_cli_cancel_req()) Skipped 2 previous similar messages Oct 11 12:59:47 maid-mds2 kernel: LustreError: 22044:0:(ldlm_request.c:1799:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 Oct 11 12:59:47 maid-mds2 kernel: LustreError: 22044:0:(ldlm_request.c:1799:ldlm_cli_cancel_list()) Skipped 2 previous similar messages Oct 11 12:59:47 maid-mds2 kernel: Lustre: MGS has stopped. Oct 11 12:59:47 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 12:59:53 maid-mds2 kernel: Lustre: 22044:0:(client.c:1778:ptlrpc_expire_one_request()) @@@ Request x1382295520936141 sent from MGC10.37.248.100@o2ib1 to NID 0@lo has timed out for slow reply: [sent 1318352387] [real_sent 1318352387] [current 1318352393] [deadline 6s] [delay 0s] req@ffff81082af48800 x1382295520936141/t0(0) o-1->MGS@MGC10.37.248.100@o2ib1_0:26/25 lens 192/192 e 0 to 1 dl 1318352393 ref 2 fl Rpc:XN/ffffffff/ffffffff rc 0/-1 Oct 11 12:59:53 maid-mds2 kernel: Lustre: 22044:0:(client.c:1778:ptlrpc_expire_one_request()) Skipped 2 previous similar messages Oct 11 12:59:53 maid-mds2 kernel: Lustre: server umount lustre-MDT0000 complete Oct 11 12:59:53 maid-mds2 kernel: Lustre: Skipped 2 previous similar messages Oct 11 13:00:25 maid-mds2 kernel: LDISKFS-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended Oct 11 13:00:25 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 13:00:25 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 13:00:25 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 13:00:25 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 13:00:25 maid-mds2 kernel: LDISKFS-fs (dm-0): mounted filesystem with ordered data mode Oct 11 13:00:25 maid-mds2 kernel: JBD: barrier-based sync failed on dm-3 - disabling barriers Oct 11 13:00:25 maid-mds2 kernel: LustreError: 22197:0:(genops.c:309:class_newdev()) Device ecc-MDT0000 already exists, won't add Oct 11 13:00:25 maid-mds2 kernel: LustreError: 22197:0:(obd_config.c:327:class_attach()) Cannot create device ecc-MDT0000 of type echo_client : -17 Oct 11 13:00:25 maid-mds2 kernel: Unable to handle kernel NULL pointer dereference at 0000000000000008 RIP: Oct 11 13:00:25 maid-mds2 kernel: [] :obdecho:echo_md_handler+0xc1/0x1970 Oct 11 13:00:25 maid-mds2 kernel: PGD bfe60f067 PUD b6b66b067 PMD 0 Oct 11 13:00:25 maid-mds2 kernel: Oops: 0000 [1] SMP Oct 11 13:00:25 maid-mds2 kernel: last sysfs file: /block/sda/queue/max_sectors_kb Oct 11 13:00:25 maid-mds2 kernel: CPU 9