[LU-12275] Client-side file data encryption

• Type: New Feature
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.14.0
• Affects Version/s: Lustre 2.14.0
• Labels: encryption sec
• Assignee: Sebastien Buisson
• Reporter: Sebastien Buisson

Description

This ticket is a place-holder to describe work to be done for client-side encryption.

The high-level requirements are the following:

• encrypt file content
• encrypt file name
• have a master key for encryption
  • per-file encryption key derived from master key
  • file data is no longer accessible after file is deleted
• able to change the user key without re-encrypting files
• deny access to encrypted data when master key is removed from memory on the client
• work in "batch scheduler" mode

We are proposing to address these requirements by:

• conforming to fscrypt kernel API
  • https://git.kernel.org/pub/scm/linux/kernel/git/tytso/fscrypt.git/tree/Documentation/filesystems/fscrypt.rst
  • https://git.kernel.org/pub/scm/linux/kernel/git/tytso/fscrypt.git/tree/fs/crypto
  • current users are ext4, F2FS, and UBIFS
  • https://git.kernel.org/pub/scm/linux/kernel/git/tytso/fscrypt.git/tree/include/linux/fscrypt.h
  • need to use newly pushed policy v2
  • https://patchwork.kernel.org/patch/10806461/
  • https://www.spinics.net/lists/linux-fscrypt/msg01357.html
• relying on ext4 encryption principles
  • file system block size = system page size
  • each filesystem block is encrypted independently in a separate block
  • pages in the page cache always contain clear text data
• mutualizing code infrastructure with compression work
  • same kind of operations, at same code locations

So the workflow would be the following:

• applications see clear text
• data is encrypted before being sent to servers
  • then remain untouched
• data is decrypted upon receipt from servers
  • untouched before that
• servers only see encrypted data
  • but do not need to be aware of it
• only client nodes have access to encryption keys

Further details will be added as the feature design makes progress.

Issue Links

• is related to: LU-7371 Wrong read length over isize
• is related to: LU-14045 Fix O_DIRECT and encrypted files
• is related to: LU-14149 FIEMAP should set FIEMAP_EXTENT_DATA_ENCRYPTED for fscrypt files
• is related to: LU-16085 Ubuntu 22.04 sanityn test_106c: supported stx_attributes: got '2030', expected '30'
• is related to: LU-16091 Set S_ENCRYPTED flag on OST objects for encrypted files
• is related to: LU-13717 Client-side encryption - support file name encryption
• is related to: LU-16091 Set S_ENCRYPTED flag on OST objects for encrypted files
• is related to: LUDOC-477 Add documentation for client-side encryption
• is related to: LU-15003 use client enc_pool for fscrypt
• is related to: LU-14677 lfs migrate/mirror of encrypted files

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50140/
Subject: LU-12275 sec: remove bio functions in fscrypt compat
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: 6ce5b0bc881389003e90d1201d468bc099251ada

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50023/
Subject: LU-12275 sec: remove bio functions in fscrypt compat
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d328818a456daf30c20c8df0aa0be9dd2a2b6a9e

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50140
Subject: LU-12275 sec: disable bio functions on client
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 546cb30c00fb1db1af59cb88396e84e362370dcf

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50023
Subject: LU-12275 sec: disable bio functions on client
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0ebb7e37b85f8e881985bdcb45d3e16ace37c1f0

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49828/
Subject: LU-12275 tests: skip new nodemap params on old MGS
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 93230059abe9dfe39a8b72cb8fc31bab1cadc7b6

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49828
Subject: LU-12275 tests: skip new nodemap params on old MGS
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f69c3cd23c4d6e128d95de781f1db5cfe68dec6f

[Peter Jones]

I believe that everything planned for 2.14 has now landed

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/39617/
Subject: LU-12275 sec: verify dir is empty when setting enc policy
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e8f74fb0f5c9306ee5a099133799e03e09ca8e47

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/39315/
Subject: LU-12275 sec: encryption with different client PAGE_SIZE
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ac5fcdce025b4825500c0308d89dfdab1faece51

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/39558/
Subject: LU-12275 sec: ldiskfs not aware of client-side encryption
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ad444ed9836320c6ae8b770ff96edd6b0fe4f0d4