[LU-4450] Not able to mount mdt device

Tue, 14 Jan 2014 02:14:28 +0000

I ran
tune2fs -O ^quota /dev/nbp7-vg/mdt7
tune2fs -O quota /dev/nbp7-vg/mdt7

After which the mdt device is not mounting.

nbp7-mds1 login: LDISKFS-fs (dm-1): recovery complete
LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. quota=on. Opts:
LDISKFS-fs (dm-2): warning: mounting fs with errors, running e2fsck is recommended
LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. quota=on. Opts:
LustreError: 137-5: nbp7-MDT0000_UUID: not available for connect from 10.151.51.135@o2ib (no target)
LustreError: 137-5: nbp7-MDT0000_UUID: not available for connect from 10.151.32.217@o2ib (no target)
LustreError: Skipped 3 previous similar messages
Lustre: nbp7-MDT0000: Not available for connect from 10.151.56.154@o2ib (not set up)
Lustre: nbp7-MDT0000: Not available for connect from 10.151.29.224@o2ib (not set up)
Lustre: Skipped 9 previous similar messages
Lustre: nbp7-MDT0000: used disk, loading
Lustre: 5049:0:(mdt_handler.c:4960:mdt_process_config()) For interoperability, skip this mdt.group_upcall. It is obsolete.
Lustre: 5049:0:(mdt_handler.c:4960:mdt_process_config()) For interoperability, skip this mdt.quota_type. It is obsolete.
LDISKFS-fs error (device dm-2): ldiskfs_mb_check_ondisk_bitmap: on-disk bitmap for group 20corrupted: 2107 blocks free in bitmap, 2105 - in gd

Aborting journal on device dm-2-8.
LDISKFS-fs (dm-2): Remounting filesystem read-only
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: IO failure
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_free_blocks: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_reserve_inode_write: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_truncate: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_reserve_inode_write: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_orphan_del: Journal has aborted
LDISKFS-fs error (device dm-2) in ldiskfs_reserve_inode_write: Journal has aborted
LustreError: 5049:0:(llog.c:159:llog_cancel_rec()) nbp7-OST0020-osc-MDT0000: fail to write header for llog #0x2:1#00000000: rc = -30
LustreError: 4960:0:(osd_handler.c:738:osd_trans_commit_cb()) transaction @0xffff880ff9a32ac0 commit error: 2
LustreError: 5049:0:(osp_sync.c:1031:osp_sync_init()) nbp7-OST0020-osc-MDT0000: can't initialize llog: rc = -30
LustreError: 5049:0:(obd_config.c:572:class_setup()) setup nbp7-OST0020-osc-MDT0000 failed (-30)
LustreError: 5049:0:(obd_config.c:1550:class_config_llog_handler()) MGC10.151.27.38@o2ib: cfg command failed: rc = -30
Lustre: cmd=cf003 0:nbp7-OST0020-osc-MDT0000 1:nbp7-OST0020_UUID 2:10.151.27.45@o2ib
LustreError: 15c-8: MGC10.151.27.38@o2ib: The configuration from log 'nbp7-MDT0000' failed (-30). This may be the result of communication errors between this node and the MGS, a b.
LustreError: 4950:0:(obd_mount_server.c:1253:server_start_targets()) failed to start server nbp7-MDT0000: -30
LustreError: 4950:0:(obd_mount_server.c:1695:server_fill_super()) Unable to start targets: -30
LustreError: 4950:0:(obd_mount_server.c:844:lustre_disconnect_lwp()) nbp7-MDT0000-lwp-MDT0000: Can't end config log nbp7-client.
LustreError: 4950:0:(obd_mount_server.c:1422:server_put_super()) nbp7-MDT0000: failed to disconnect lwp. (rc=-2)
Lustre: Failing over nbp7-MDT0000
Lustre: nbp7-MDT0000: Not available for connect from 10.151.43.177@o2ib (stopping)
Lustre: Skipped 151 previous similar messages
LustreError: 137-5: nbp7-MDT0000_UUID: not available for connect from 10.151.32.62@o2ib (no target)
LustreError: Skipped 81 previous similar messages
VFS: cannot write quota structure on device dm-2 (error -30). Quota may get out of sync!
VFS: cannot write quota structure on device dm-2 (error -30). Quota may get out of sync!
LDISKFS-fs error (device dm-2): ldiskfs_put_super: Couldn't clean up the journal
Lustre: server umount nbp7-MDT0000 complete

Please advice next step.

Whamcloud Community JIRA

[LU-4450] Not able to mount mdt device