[LU-16259] HLD for encrypted backup/restore Created: 21/Oct/22  Updated: 20/Apr/23  Resolved: 30/Jan/23

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Minor
Reporter: Sebastien Buisson Assignee: Sebastien Buisson
Resolution: Fixed Votes: 0
Labels: encryption, sec

Attachments: PDF File HLD_backup_restore_enc_files_v0.3.pdf    
Issue Links:
Related
is related to LU-16374 Implement backup/restore of encrypted... Open
Rank (Obsolete): 9223372036854775807
Epic Link: Client side Encrypted backup/restore

 Description   

The use cases for encrypted files backup/restore we want to address are:

  • backup/restore of MDT and OST directly at the ldiskfs level, to be able to migrate to a newly-formatted MDT or OST filesystem.
    In this case we are considering the use of a patched tar to carry out specific operations if an encrypted file is detected. This consequently requires that encrypted files are properly flagged at the ldiskfs level.
  • backup/restore of regular files at the Lustre client level, without the encryption key so that no clear text copy of encrypted files are made.
    Again in this case we are considering the use of a modified tar that would carry out specific operations on encrypted files, and passing the O_FILE_ENC | O_DIRECT flags to be able to access to raw content without the encryption key.
  • Lustre/HSM without the encryption key so that no clear text copy of encrypted files are made.
    This will consist in adjusting the lhsmtool_posix utility's behavior for encrypted files.

We are considering the use of a 'virtual' xattr named for instance "security.encdata". Fetching or setting this xattr would have a special meaning for llite and ldiskfs, and this xattr could be filled with useful information such as clear text size (or preferably a delta between encrypted file size and clear text file size in order to save space in the xattr), encryption context and encrypted name. Setting this security.encdata xattr will be limited to the case when the file does not have an encryption context yet, to avoid users changing the encryption context after a file is created.

For testing and development purposes, we want to create a new lfs command to trigger backup and restore on Lustre files whose path is given as input. That will leverage the internal mechanisms mentioned above, and could also be used as-is for specific workflows such as moving encrypted files between file systems without decrypt/encrypt.



 Comments   
Comment by Sebastien Buisson [ 21/Oct/22 ]

The HLD can be accessed at:

https://datadirectnetworks-my.sharepoint.com/:w:/g/personal/sbuisson_ddn_com/EeWD3Q7Ku69Anntda03QPDUBs6oxRCxtlxELM7xxy-S1qQ

Everyone should have revision access rights.

Generated at Sat Feb 10 03:25:25 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.