Details
-
Improvement
-
Resolution: Fixed
-
Major
-
None
-
None
-
8773
Description
The Lustre backup/restore documentation describes how to do a backup/restore using both "tar" and "dd" for ldiskfs devices. However, the preferred mechanism for backing up and restoring ZFS filesystems is via the "zfs send" and "zfs recv" mechanism. This will preserve the ZFS dnode numbers, and all of the FID->dnode mappings in the OI files. The zfs dump/restore functionality can also be used to do incremental backups, and keep two physically-separate devices in relatively close synchronization for disaster-recovery and other purposes.
It is also not currently possible to do a file-level backup from ldiskfs and restore this into a new ZFS filesystem because there is no OI Scrub facility for ZFS OSDs, so that should be documented as unsupported until LU-7585 is resolved.
Attachments
Issue Links
Activity
Joseph Gmitter (joseph.gmitter@intel.com) merged in patch https://review.whamcloud.com/30172/
Subject: LUDOC-161 zfs: Backup/Restore of ZFS backend filesystems
Project: doc/manual
Branch: master
Current Patch Set:
Commit: 3eba9c153757b350882c28161e45b4e5815617b2
Joseph Gmitter (joseph.gmitter@intel.com) uploaded a new patch: https://review.whamcloud.com/30172
Subject: LUDOC-161 zfs: Backup/Restore of ZFS backend filesystems
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 843d957d7553ade81404dd3fdbefc6240b32d11a
Please consider to update Lustre manual according to above three comments. Thanks!
If you want to migrate the system from ldiskfs to ZFS, please backup your system as following:
1) Before umount the target (MDT or OST), please enable index backup on the target. For example:
lctl set_param osd-ldiskfs.${fsname}-MDT0000.index_backup=1
Or you can write the proc interface on the target directly:
echo 1 > /proc/fs/lustre/osd-ldiskfs/${fsname}-MDT0000/index_backup
2) Umount the target.
Then you can follow the standard steps to backup the target normally via "tar" and "getfattr" as described in the Lustre manual 18.3.
The steps for restore ZFS backend via ZPL by 'tar':
1) Create new pool for the target if necessary, then reformat new Lustre FS with "--replace" parameter. For example:
mkfs.lustre --mgs --fsname=${fsname} --mdt --index=0 --replace --param=sys.timeout=20 --backfstype=zfs --reformat ${fsname}-mdt1/mdt1 ${target_device}
2) Enable "canmount" property on the target FS. For example:
zfs set canmount=on ${fsname}-mdt1/mdt1
3) Mount the target as 'zfs'. For example:
zfs mount ${fsname}-mdt1/mdt1
4) Retore the data. For example:
tar jxf /tmp/target.tar.bz2 --xattrs --xattrs-include="trusted.*" -C /${fsname}-mdt1/mdt1/
5) Remove stale OIs and index objects. For example:
cd /${fsname}-mdt1/mdt1 && rm -rf oi.* OI_* lfsck_* LFSCK && sync && cd -
6) Umount the target.
7) (optional) If the restored system has different NID as the backup system, please change NID. For detail, please refer to Lustre manual 14.5. For example:
mount -t lustre -o nosvc ${fsname}-mdt1/mdt1 $mount_point
lctl replace_nids ${fsname}-MDTxxxx $new_nids
...
umount $mount_point
8) Mount the target as "lustre". Usually, we will use "-o abort_recov" option to skip unnecessary recovery. For example:
mount -t lustre -o abort_recov ${fsname}-mdt1/mdt1 $mount_point
The osd-zfs can detect the restore automatically when mount the target, then trigger OI scrub to rebuild the OIs and index objects asynchronously at background. You can check the OI scrub status. For example:
lctl get_param -n osd-zfs.${fsname}-${target}.oi_scrub
Or you can read the proc interface on the target directly:
cat /proc/fs/lustre/osd-zfs/${fsname}-${target}.oi_scrub
If you want to restore the system from ldiskfs-based backup, please follow the same steps.
The steps for backup ZFS backend var ZPL by 'tar':
1) Before umount the target (MDT or OST), please enable index backup on the target. For example:
lctl set_param osd-zfs.${fsname}-MDT0000.index_backup=1
Or you can write the proc interface on the target directly:
echo 1 > /proc/fs/lustre/osd-zfs/${fsname}-MDT0000/index_backup
2) Umount the target.
3) Import the pool for the target if it is exported during the step 2). For example:
zpool import lustre-mdt1 [-d ${target_device_dir}]
4) Enable "canmount" property on the target FS. For example:
zfs set canmount=on ${fsname}-mdt1/mdt1
You also can specify the "mountpoint" property. By default, it will be /${fsname}-mdt1/mdt1
5) Mount the target as 'zfs'. For example:
zfs mount ${fsname}-mdt1/mdt1
6) Backup the data. For example:
tar jcf /tmp/target.tar.bz2 --xattrs --xattrs-include="trusted.*" -C /${fsname}-mdt1/mdt1/ .
7) Umount the target and export the pool if you want.
Please save the /tmp/target.bz2 and /tmp/target.ea. If you want to migrate system from ZFS to ldiskfs, please backup the system as the same steps.
Hi Andreas,
I would be glad to take a crack at updating the Lustre user manual to document the procedure to do a device level backup of a ZFS MDT or OST.
Hi Tom,
Based on your LUG presentation today about using ZFS send/recv for migration, the process to do a backup of the ZFS MDT (or OST). Would you be interested to take a crack st updating the Lustre user manual to document the procedure to do a device level backup of a ZFS MDT or OST? It should be noted that rsync is not a workable solution for ZFS until the ZFS OI scrub is implemented.
Included in the 2.11 manual.