==> /tmp/serial_fre1133_out <== [ 133.785567] Lustre: DEBUG MARKER: excepting tests: 61d 33a 33b 62 [ 136.017539] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 137.660707] Lustre: DEBUG MARKER: == replay-single test 14: open(O_CREAT), unlink |X| close == 13:32:14 (1448458334) [ 138.215352] LustreError: 2815:0:(osd_handler.c:1538:osd_ro()) *** setting lustre-MDT0000 read-only *** [ 138.217756] Turning device vdb (0xfd00010) read-only [ 138.330298] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 138.561112] Lustre: Failing over lustre-MDT0000 [ 139.916744] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 192.168.111.34@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 139.920231] LustreError: Skipped 1 previous similar message [ 143.835424] LustreError: 137-5: lustre-MDT0000_UUID: not available for connect from 192.168.111.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 144.607150] Lustre: 2858:0:(client.c:1994:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1448458335/real 1448458335] req@ffff8800bfcdc900 x1518818569487004/t0(0) o251->MGC192.168.111.33@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1448458341 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1 [ 144.619791] Removing read-only on unknown block (0xfd00010) [ 144.649835] Lustre: server umount lustre-MDT0000 complete [ 155.322712] LDISKFS-fs (vdb): recovery complete [ 155.323961] LDISKFS-fs (vdb): mounted filesystem with ordered data mode. Opts: user_xattr,user_xattr,errors=remount-ro,no_mbcache [ 155.343169] Lustre: MGS: Connection restored to 50c7b812-69d0-e870-799c-9944b0328123 (at 0@lo) [ 155.346273] Lustre: Skipped 1 previous similar message [ 155.384773] LustreError: 2968:0:(lquota_disk.c:284:lquota_disk_glb_find_create()) lustre-MDT0000-osd: failed to setup index operations for [0x200000006:0x20000:0x0] rc:-9 [ 155.394289] LustreError: 2968:0:(qmt_pool.c:499:qmt_pool_prepare()) lustre-QMT0000: failed to create glb index copy for usr type (-9) [ 155.403144] LustreError: 2968:0:(obd_config.c:595:class_detach()) OBD device 7 not attached [ 155.443455] LustreError: 2968:0:(obd_config.c:575:class_setup()) setup lustre-MDT0000 failed (-9) [ 155.448411] LustreError: 2968:0:(obd_config.c:1663:class_config_llog_handler()) MGC192.168.111.33@tcp: cfg command failed: rc = -9 [ 155.455520] Lustre: cmd=cf003 0:lustre-MDT0000 1:lustre-MDT0000_UUID 2:0 3:lustre-MDT0000-mdtlov 4:f [ 155.455520] [ 155.462762] LustreError: 15c-8: MGC192.168.111.33@tcp: The configuration from log 'lustre-MDT0000' failed (-9). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information. [ 155.475959] LustreError: 2921:0:(obd_mount_server.c:1306:server_start_targets()) failed to start server lustre-MDT0000: -9 [ 155.480076] LustreError: 2921:0:(obd_mount_server.c:1794:server_fill_super()) Unable to start targets: -9 [ 155.483664] LustreError: 2921:0:(obd_config.c:622:class_cleanup()) Device 5 not setup [ 160.705266] general protection fault: 0000 [#1] SMP [ 160.706212] Modules linked in: osp(OF) mdd(OF) lod(OF) mdt(OF) lfsck(OF) mgs(OF) mgc(OF) osd_ldiskfs(OF) lquota(OF) fid(OF) fld(OF) ksocklnd(OF) ptlrpc(OF) obdclass(OF) lnet(OF) sha512_generic libcfs(OF) ldiskfs(OF) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ppdev virtio_balloon virtio_net serio_raw pcspkr parport_pc parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd sunrpc ext4 mbcache jbd2 ata_generic pata_acpi virtio_blk cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper ttm ata_piix drm virtio_pci virtio_ring i2c_core virtio libata floppy [ 160.706212] CPU: 0 PID: 1542 Comm: obd_zombid Tainted: GF O-------------- 3.10.0-229.20.1.el7_lustremaster_master__77.x86_64 #1 [ 160.706212] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007 [ 160.706212] task: ffff8800db1271c0 ti: ffff8800d412c000 task.ti: ffff8800d412c000 [ 160.706212] RIP: 0010:[] [] tgt_client_free+0x1c8/0x4e0 [ptlrpc] [ 160.706212] RSP: 0018:ffff8800d412fdd0 EFLAGS: 00010206 [ 160.706212] RAX: 5a5a5a5a5a5a5a5a RBX: ffff8800c5f73f00 RCX: 0000000180200009 [ 160.706212] RDX: 0000000000000000 RSI: ffffea0003183e40 RDI: 0000000040000000 [ 160.706212] RBP: ffff8800d412fdf8 R08: ffff8800c60f9780 R09: 0000000180200009 [ 160.706212] R10: ffffea0003183e40 R11: ffffffffa08629b8 R12: ffff8800c5f73c00 [ 160.706212] R13: ffff8801190580b0 R14: ffff8800c5f73ea8 R15: ffff8800c5f73f00 [ 160.706212] FS: 0000000000000000(0000) GS:ffff88011fc00000(0000) knlGS:0000000000000000 [ 160.706212] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 160.706212] CR2: 00007fbc798cb000 CR3: 00000000c9fbd000 CR4: 00000000000006f0 [ 160.706212] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 160.706212] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 160.706212] Stack: [ 160.706212] ffff8800c5f73c00 ffff8800c5f73c80 00001ffffffffffe 0000370000000000 [ 160.706212] 0000000000000000 ffff8800d412fe10 ffffffffa0bc261d ffff8800c01f4bc8 [ 160.706212] ffff8800d412fe50 ffffffffa058446a ffff8800c5f73c00 ffff8800db1271c0 [ 160.706212] Call Trace: [ 160.706212] [] mdt_destroy_export+0x6d/0x1e0 [mdt] [ 160.706212] [] obd_zombie_impexp_cull+0x5ba/0xa80 [obdclass] [ 160.706212] [] obd_zombie_impexp_thread+0x6d/0x1c0 [obdclass] [ 160.706212] [] ? wake_up_state+0x20/0x20 [ 160.706212] [] ? obd_zombie_impexp_cull+0xa80/0xa80 [obdclass] [ 160.706212] [] kthread+0xcf/0xe0 [ 160.706212] [] ? kthread_create_on_node+0x140/0x140 [ 160.706212] [] ret_from_fork+0x58/0x90 [ 160.706212] [] ? kthread_create_on_node+0x140/0x140 [ 160.706212] Code: 00 00 49 c7 84 24 d0 02 00 00 00 00 00 00 85 d2 78 4b 4d 85 ed 0f 84 d2 01 00 00 49 8b 85 18 03 00 00 48 85 c0 0f 84 c2 01 00 00 0f b3 10 19 d2 85 d2 0f 84 7d 02 00 00 41 f6 84 24 6f 01 00 [ 160.706212] RIP [] tgt_client_free+0x1c8/0x4e0 [ptlrpc] [ 160.706212] RSP [ 160.821948] ---[ end trace 2f78023bbc8ae54b ]--- [ 160.826503] Kernel panic - not syncing: Fatal exception [ 160.827494] drm_kms_helper: panic occurred, switching back to text console