[LU-14677] lfs migrate/mirror of encrypted files

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.15.0
• Affects Version/s: Lustre 2.14.0, Lustre 2.15.0
• Assignee: Sebastien Buisson
• Reporter: Andreas Dilger

Description

It should be possible to use "lfs migrate" and "lfs mirror extend/resync" of encrypted files, even if the process does not have access to the key. There is no need for these operations to actually understand the file data, only to read the data in whole-chunk units and then write it to a new file copy/mirror before updating the file layout.

Barriers to handling this properly are:

• need to be able to open() the encrypted file without a key, since this currently returns -ENOKEY to callers, so that they don't accidentally try to read/write unintelligible data. This could be handled with a special open flag (e.g. something similar to O_LOV_DELAY_CREATE for tools that know what they are doing. This should be done with an llapi_file_open_fscrypt() helper to isolate the logic in case it needs to be changed in the future.
• data tools should always read/write the full 4KB chunk of encrypted data. IMHO, this should imply that the file size is always reported to userspace rounded up to the next 4KB chunk size. That shouldn't affect processes that have the key, and helps somewhat to avoid fingerprinting files based on their size.

For file backup/restore, the tools would also need to be able to read/save/restore the unique salt/nonce stored with the file so that the file can later be decrypted again. Special considerations are needed when restoring an encrypted file, since it is typically not possible to create "unencrypted" files in a directory with an encryption key, and similarly setting an encryption key on a non-empty directory is not possible. As such backup/restore of cyphertext files should be handled in a separate ticket. In the meantime, it would be possible to backup restore encrypted files in plaintext (presumably to a medium that is itself encrypted) if the backup tools have access to a master key added to each fscrypt directory key).

Issue Links

• is related to: LU-16091 Set S_ENCRYPTED flag on OST objects for encrypted files
• is related to: LU-12275 Client-side file data encryption
• is related to: LU-13717 Client-side encryption - support file name encryption
• is related to: LU-15790 Fix mirror read/write in Test Plan for File Name Encryption Feature
• is related to: LU-16374 Implement backup/restore of encrypted files

Comments

[Peter Jones]

Landed for 2.15

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/44957/
Subject: LU-14677 sec: remove MIGRATION_ compatibility defines
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e42d2d67d3a0dcc726d1424d3158b6f649b5abd7

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/44024/
Subject: LU-14677 sec: no encryption key migrate/extend/resync/split
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: fdbf2ffd41fa5660782d5ca8489ec2eb644c8113

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/44101/
Subject: LU-14677 sec: do not expose security.c to listxattr/getxattr
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: efb66de719329ce4d96b40f00ad592cca1e432fd

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/44198/
Subject: LU-14677 llite: move env contexts to ll_inode_info level
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 957e7de61ec129013ba0df90c3abe64ff024e438

[Gerrit Updater]

"Sebastien Buisson <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/44957
Subject: LU-14677 sec: change MIGRATION_ flags to LLAPI_MIGRATION_
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f0b4d394a70eab516c6a2320d4c3a7cacee34ebc

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43878/
Subject: LU-14677 sec: migrate/extend/split on encrypted file
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 09c558d16f0a80f436522edde89367c088fe2055

[Gerrit Updater]

Sebastien Buisson ([e-mail]) uploaded a new patch: https://review.whamcloud.com/44198
Subject: LU-14677 llite: move env contexts to ll_inode_info level
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 932007c91333117b7b0905ce5601aafc9b3bdd4e

[Gerrit Updater]

Sebastien Buisson ([e-mail]) uploaded a new patch: https://review.whamcloud.com/44101
Subject: LU-14677 sec: do not expose security.c to listxattr/getxattr
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a7c4cdbd42f7ce599ccfe3aa4160f6eb44cc32f1

[Gerrit Updater]

Sebastien Buisson ([e-mail]) uploaded a new patch: https://review.whamcloud.com/44024
Subject: LU-14677 sec: no encryption key migrate/extend/resync/split
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5b6879499b5973f12e26624ee5945adbac097129