[LU-11512] DL-SNAP: Directory Level Snapshot

• Type: New Feature
• Resolution: Won't Fix
• Priority: Minor
• Affects Version/s: Lustre 2.6.0
• Labels: patch
• Assignee: Tatsushi Takamura
• Reporter: Tatsushi Takamura

Description

DL-SNAP is a feature designed for directory level file backups.

It is implemented on top of lustre ldiskfs without modification of ext4 disk format, though we set a special original flag to FEATURE of the superblock in order to check whether DL-SNAP is enabled.

The feature makes execution of e2fsck command failed although no modification to ext2 file system structure.

 

DL-SNAP uses COW(Copy On Write) mechanism to reduce backup time and storage usage.  When we create a snapshot, only inodes are created. At this point, new data blocks are not allocated on OST.  Data blocks of snapshot are allocated when original file is modified.

 

Not only root users but also ordinary users can create snapshots. Users can create a snapshot of a directory using lfs command with snapshot option, and restore files by usual process such as cp command.

 

For more information, please check the presentation at LUG2016.

Comments

[Tatsushi Takamura]

We didn't understand that adding original code to ldiskfs is not acceptable, because there are some patches(lustre/kernel_patches/patches/) modifying original kernel source.
This DL-SNAP code is impossible to be merged into Ext4 mainline and it is impossible to realize DL-SNAP feature without adding code to ldiskfs.
So we think it is impossible to be merged into lustre.

For future reference, could you tell us the rules(which sources are acceptable or not to modify), difference in contents or polices of "ldiskfs/kernel_patches/patches/" and "lustre/kernel_patches/patches/".
For example, can we create patches(not merged in mainline) which modify kernel code in "lustre/kernel_patches/patches/"?

[Li Xi]

I've been looking into the Ext4 patch a little bit. Please correct me if I am wrong Tatsushi-san, but I think on Ext4 level, recurisve subdirectory snapshot is not supported. It might be implemented on Lustre level? I have a feeling that data on MDT might causes some difficulty for this snapshot implementation.

[Andreas Dilger]

Thank you for contributing the patch. Is there any kind of design documentation or description that was written before or during development? Even if it is in Japanese, we can try machine translation to try and understand the implementation better.

[Wang Shilong (Inactive)]

Still not look into the detailed codes, but from PPT, a quick question:

Is DL-SNAP recursive for sub dir?

[Gerrit Updater]

Tatsushi Takamura ([e-mail]) uploaded a new patch: https://review.whamcloud.com/33555
Subject: LU-11512 ldiskfs: DL-SNAP Directory Level Snapshot
Project: fs/lustre-release
Branch: b2_6
Current Patch Set: 1
Commit: 303f237c5e116ac271f03db3cc1a3aeb2568b7f0

[Tatsushi Takamura]

Sorry for the late reply. I have attached dl_snapshot patch to this ticket.

dl_snapshot.patch

[Andreas Dilger]

This ticket has the "patch" label, but I don't see a patch attached to this ticket or in Gerrit?