Details
-
Bug
-
Resolution: Done
-
Blocker
-
None
-
Lustre 2.5.3
-
None
-
Centos 6, Lustre from llnl chaos branch
-
3
-
9223372036854775807
Description
The MDT for one of our filesystems is full, and it's not possible to delete any files, rendering the filesystem unusable from the users point of view.
It's possible to manually track files that could be deleted via fid to ZFS objects on the disk. But we haven't found a way to delete objects via zdb. A recovery procedure using something like that would probably be good to have if more people run in to this.
Given that it's almost Christmas vacation, so lets keep this simple and low risk. I've thrown some more disks into the MDS. Given that the filesystem with problems looks like this:
lustre-mdt0 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
mds9_sdm-mdt_fouo6_sdm ONLINE 0 0 0
mds9_sdn-mdt_fouo6_sdn ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
mds9_sdo-mdt_fouo6_sdo ONLINE 0 0 0
mds9_sdp-mdt_fouo6_sdp ONLINE 0 0 0
Would it be safe (and fix the problem) to expand it by adding another mirror?:
zpool add lustre-mdt0 mirror /dev/exp_sdq/mdt_fouo6exp_sdq /dev/exp_sdt/mdt_fouo6exp_sdt
(This is probably the same issue as LU-8856, so feel free to merge them if it makes sense.)