Apr 6 16:30:36 service360 kernel: Lustre: DEBUG MARKER: -----============= acceptance-small: replay-single ============----- Fri Apr 6 16:30:36 PDT 2012 Apr 6 16:30:37 service360 kernel: Lustre: DEBUG MARKER: only running test 0a Apr 6 16:30:37 service360 kernel: Lustre: DEBUG MARKER: excepting tests: Apr 6 16:30:37 service360 kernel: Lustre: DEBUG MARKER: skipping tests SLOW=no: 1 2 3 4 6 12 16 44a 44b 65 66 67 68 Apr 6 16:30:38 service360 kernel: Lustre: DEBUG MARKER: Using TIMEOUT=20 Apr 6 16:30:39 service360 kernel: Lustre: 4678:0:(debug.c:326:libcfs_debug_str2mask()) You are trying to use a numerical value for the mask - this will be deprecated in a future release. Apr 6 16:30:39 service360 kernel: Lustre: DEBUG MARKER: == test 0a: empty replay == 16:30:39 Apr 6 16:30:40 service360 kernel: LustreError: 4707:0:(osd_handler.c:938:osd_ro()) *** setting device osd-ldiskfs read-only *** Apr 6 16:30:40 service360 kernel: Turning device sdb (0x800011) read-only Apr 6 16:30:40 service360 kernel: Lustre: DEBUG MARKER: mds REPLAY BARRIER on lustre-MDT0000 Apr 6 16:30:40 service360 kernel: Lustre: Failing over lustre-MDT0000 Apr 6 16:30:40 service360 kernel: Write to readonly device sdb (0x800011) bi_flags: f000000000000001, bi_vcnt: 1, bi_idx: 0, bi->size: 4096, bi_cnt: 2, bi_private: ffff8803ec324880 Apr 6 16:30:40 service360 kernel: Lustre: 4722:0:(quota_master.c:793:close_quota_files()) quota[0] is off already Apr 6 16:30:40 service360 kernel: Lustre: 4722:0:(quota_master.c:793:close_quota_files()) quota[1] is off already Apr 6 16:30:40 service360 kernel: Lustre: Failing over mdd_obd-lustre-MDT0000 Apr 6 16:30:40 service360 kernel: Lustre: mdd_obd-lustre-MDT0000: shutting down for failover; client state will be preserved. Apr 6 16:30:40 service360 kernel: Write to readonly device sdb (0x800011) bi_flags: f000000000000001, bi_vcnt: 1, bi_idx: 0, bi->size: 4096, bi_cnt: 2, bi_private: ffff8803ec3248e8 Apr 6 16:30:40 service360 kernel: LustreError: 4722:0:(ldlm_request.c:1172:ldlm_cli_cancel_req()) Got rc -108 from cancel RPC: canceling anyway Apr 6 16:30:40 service360 kernel: LustreError: 4722:0:(ldlm_request.c:1799:ldlm_cli_cancel_list()) ldlm_cli_cancel_list: -108 Apr 6 16:30:40 service360 kernel: LustreError: 4138:0:(mgs_handler.c:678:mgs_handle()) lustre_mgs: operation 251 on unconnected MGS Apr 6 16:30:40 service360 kernel: LustreError: 4138:0:(ldlm_lib.c:2129:target_send_reply_msg()) @@@ processing error (-107) req@ffff8803ecf9d050 x1398543498608695/t0(0) o-1->@:0/0 lens 192/0 e 0 to 0 dl 1333755170 ref 1 fl Interpret:/ffffffff/ffffffff rc -107/-1 Apr 6 16:30:40 service360 kernel: LustreError: 11-0: an error occurred while communicating with 0@lo. The mgs_disconnect operation failed with -107 Apr 6 16:30:41 service360 kernel: Lustre: MGS has stopped. Apr 6 16:30:43 service360 kernel: Write to readonly device sdb (0x800011) bi_flags: f000000000000001, bi_vcnt: 1, bi_idx: 0, bi->size: 4096, bi_cnt: 2, bi_private: ffff8803ec2ea950 Apr 6 16:30:43 service360 kernel: Write to readonly device sdb (0x800011) bi_flags: f000000000000001, bi_vcnt: 1, bi_idx: 0, bi->size: 4096, bi_cnt: 2, bi_private: ffff8803ec2e7268 Apr 6 16:30:43 service360 kernel: Write to readonly device sdb (0x800011) bi_flags: f000000000000001, bi_vcnt: 1, bi_idx: 0, bi->size: 4096, bi_cnt: 2, bi_private: ffff8803ec2ea950 Apr 6 16:30:43 service360 kernel: Removing read-only on unknown block (0x800011) Apr 6 16:30:43 service360 kernel: Lustre: server umount lustre-MDT0000 complete Apr 6 16:30:43 service360 sec[2003]: SEC_EVENT |msg ldiskfs umount|target lustre-MDT0000 Apr 6 16:30:53 service360 kernel: LDISKFS-fs warning (device sdb1): ldiskfs_fill_super: extents feature not enabled on this filesystem, use tune2fs. Apr 6 16:30:54 service360 kernel: LDISKFS-fs (sdb1): recovery complete Apr 6 16:30:54 service360 kernel: LDISKFS-fs (sdb1): mounted filesystem with ordered data mode. Opts: Apr 6 16:30:54 service360 kernel: LDISKFS-fs warning (device sdb1): ldiskfs_fill_super: extents feature not enabled on this filesystem, use tune2fs. Apr 6 16:30:54 service360 kernel: LDISKFS-fs (sdb1): mounted filesystem with ordered data mode. Opts: Apr 6 16:30:54 service360 kernel: Lustre: MGS MGS started Apr 6 16:30:54 service360 kernel: Lustre: 4744:0:(sec.c:1474:sptlrpc_import_sec_adapt()) import MGC10.151.26.38@o2ib->MGC10.151.26.38@o2ib_0 netid 90000: select flavor null Apr 6 16:30:54 service360 kernel: Lustre: 4744:0:(sec.c:1474:sptlrpc_import_sec_adapt()) Skipped 2 previous similar messages Apr 6 16:30:54 service360 kernel: Lustre: 4774:0:(ldlm_lib.c:877:target_handle_connect()) MGS: connection from a66295f1-0f8b-716a-7c40-09733f3b1f52@0@lo t0 exp (null) cur 1333755054 last 0 Apr 6 16:30:54 service360 kernel: Lustre: 4774:0:(ldlm_lib.c:877:target_handle_connect()) Skipped 3 previous similar messages Apr 6 16:30:54 service360 kernel: Lustre: MGC10.151.26.38@o2ib: Reactivating import Apr 6 16:30:54 service360 kernel: Lustre: Enabling ACL Apr 6 16:30:54 service360 kernel: Lustre: lustre-MDT0000: used disk, loading Apr 6 16:30:54 service360 kernel: Lustre: 4778:0:(ldlm_lib.c:1904:target_recovery_init()) RECOVERY: service lustre-MDT0000, 2 recoverable clients, last_transno 4294967296 Apr 6 16:30:54 service360 kernel: LustreError: 4782:0:(ldlm_lib.c:1741:target_recovery_thread()) lustre-MDT0000: started recovery thread pid 4782 Apr 6 16:30:54 service360 kernel: Lustre: 4778:0:(mds_lov.c:1004:mds_notify()) MDS mdd_obd-lustre-MDT0000: add target lustre-OST0000_UUID Apr 6 16:30:54 service360 kernel: LustreError: 4784:0:(mdt_handler.c:2790:mdt_recovery()) operation 41 on unconnected MDS from 12345-10.151.25.183@o2ib Apr 6 16:30:54 service360 kernel: LustreError: 4783:0:(ldlm_lib.c:2129:target_send_reply_msg()) @@@ processing error (-107) req@ffff8803f3a3bc00 x1398543424159780/t0(0) o-1->@:0/0 lens 192/0 e 0 to 0 dl 1333755184 ref 1 fl Interpret:/ffffffff/ffffffff rc -107/-1 Apr 6 16:30:55 service360 kernel: Lustre: 4095:0:(mds_lov.c:1024:mds_notify()) MDS mdd_obd-lustre-MDT0000: in recovery, not resetting orphans on lustre-OST0000_UUID Apr 6 16:30:55 service360 kernel: Lustre: 4883:0:(debug.c:326:libcfs_debug_str2mask()) You are trying to use a numerical value for the mask - this will be deprecated in a future release. Apr 6 16:30:55 service360 kernel: Lustre: 4883:0:(debug.c:326:libcfs_debug_str2mask()) Skipped 1 previous similar message Apr 6 16:30:55 service360 kernel: Lustre: 4095:0:(mds_lov.c:1024:mds_notify()) MDS mdd_obd-lustre-MDT0000: in recovery, not resetting orphans on lustre-OST0001_UUID Apr 6 16:30:55 service360 kernel: LustreError: 4784:0:(mdt_handler.c:2790:mdt_recovery()) Skipped 1 previous similar message Apr 6 16:30:55 service360 kernel: Lustre: lustre-MDT0000: sending delayed replies to recovered clients Apr 6 16:30:55 service360 kernel: LustreError: 4774:0:(mgs_handler.c:678:mgs_handle()) lustre_mgs: operation 400 on unconnected MGS Apr 6 16:30:55 service360 kernel: Lustre: MDS mdd_obd-lustre-MDT0000: lustre-OST0000_UUID now active, resetting orphans Apr 6 16:30:55 service360 kernel: Lustre: Skipped 1 previous similar message Apr 6 16:30:56 service360 kernel: Lustre: DEBUG MARKER: ===== replay-single.sh test complete, duration 20 sec =======================