[LU-9570] Unable to mount OSTs after power event

Sun, 28 May 2017 16:18:15 +0000

We had a power event last night that impacted Atlas. We're currently unable to mount some of the OSTs as part of the bringup. We tried to mount as ldiskfs and that is not working either.
{{
[root@atlas-oss4b7 ~]# mount -t lustre /dev/mapper/atlas-ddn4b-l43 /tmp/tmpost
mount.lustre: mount /dev/mapper/atlas-ddn4b-l43 at /tmp/tmpost failed: Invalid argument
This may have multiple causes.
Are the mount options correct?
Check the syslog for more info.
[root@atlas-oss4b7 ~]# mount -t ldiskfs /dev/mapper/atlas-ddn4b-l43 /tmp/tmpost
mount: wrong fs type, bad option, bad superblock on /dev/mapper/atlas-ddn4b-l43,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so
[root@atlas-oss4b7 ~]# dmesg | tail
[ 1589.056034] LustreError: 51009:0:(osd_handler.c:6365:osd_mount()) atlas2-OST00e6-osd: can't mount /dev/mapper/atlas-ddn4b-l43: -22
[ 1589.081213] LustreError: 51009:0:(obd_config.c:578:class_setup()) setup atlas2-OST00e6-osd failed (-22)
[ 1589.120021] LustreError: 51009:0:(obd_mount.c:203:lustre_start_simple()) atlas2-OST00e6-osd setup error -22
[ 1589.150512] LustreError: 51009:0:(obd_mount_server.c:1764:server_fill_super()) Unable to start osd on /dev/mapper/atlas-ddn4b-l43: -22
[ 1589.190570] LustreError: 51009:0:(obd_mount.c:1426:lustre_fill_super()) Unable to mount (-22)
[ 1609.130825] LustreError: 51284:0:(osd_handler.c:6365:osd_mount()) atlas2-OST00e6-osd: can't mount /dev/mapper/atlas-ddn4b-l43: -22
[ 1609.158986] LustreError: 51284:0:(obd_config.c:578:class_setup()) setup atlas2-OST00e6-osd failed (-22)
[ 1609.197816] LustreError: 51284:0:(obd_mount.c:203:lustre_start_simple()) atlas2-OST00e6-osd setup error -22
[ 1609.228156] LustreError: 51284:0:(obd_mount_server.c:1764:server_fill_super()) Unable to start osd on /dev/mapper/atlas-ddn4b-l43: -22
[ 1609.268254] LustreError: 51284:0:(obd_mount.c:1426:lustre_fill_super()) Unable to mount (-22)
[root@atlas-oss4b7 ~]#
}}
The lun is still readable:
{{
[root@atlas-oss4b7 ~]# dumpe2fs -h /dev/mapper/atlas-ddn4b-l43 | head
dumpe2fs 1.42.13.wc5 (15-Apr-2016)
Filesystem volume name: atlas2-OST00e6
Last mounted on: /
Filesystem UUID: d4045c22-5929-4c58-bf58-4251bb21e07b
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent mmp flex_bg sparse_super large_file huge_file uninit_bg dir_nlink quota
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
}}
We encountered issues when trying to do an `e2fsck`:
{{
[root@atlas-oss4b7 ~]# e2fsck -n /dev/mapper/atlas-ddn4b-l43
e2fsck 1.42.13.wc5 (15-Apr-2016)
Superblock has invalid MMP magic. Fix? no
atlas2-OST00e6: ********** WARNING: Filesystem still has errors **********
}}
Systems are currently down due to this. There are multiple OSTs in this state.
Thanks,
–
Jesse

Whamcloud Community JIRA

[LU-9570] Unable to mount OSTs after power event