-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
None
-
3
-
9223372036854775807
"lfs migrate" of an encrypted file by a process that has the encryption key loaded corrupts file data beyond about the first 1MiB. Files up to 1MiB migrate intact, which is why sanity-sec test_52 (9000-byte file) passes.
Reproducer on master (2.17.59-1-g4a7bdda, single-node VM with client crypto enabled, mounted with test_dummy_encryption):
mkdir /mnt/lustre/dm for sz in 9000 65536 1048576 3145728; do f=/mnt/lustre/dm/m$sz head -c $sz /dev/urandom > /tmp/src; cp /tmp/src $f; sync lfs migrate -c 2 $f lctl set_param -n ldlm.namespaces.*osc*.lru_size=clear cmp -s /tmp/src $f && echo "$sz ok" || echo "$sz CORRUPT" done 9000 ok 65536 ok 1048576 ok 3145728 CORRUPT
In the 3MiB file, pages 276-767 (offset 1104KiB to EOF) differ from the source; pages 0-275 are intact. The source and the volatile file are both opened with O_CIPHERTEXT|O_DIRECT, but with the key loaded the direct read returns plaintext and the direct write encrypts it with the volatile file's own encryption context, before the layout swap gives the data to the original inode. "lfs mirror extend" of a 3MiB file with the key loaded likewise gave a new mirror whose content (read via "lfs mirror read -N 2") differs from the file.
The root cause is not yet known; the boundary not being a multiple of the RPC size suggests something like the encryption page pool when a direct write spans several RPCs. A related change is planned to make O_CIPHERTEXT direct IO always transfer the raw ciphertext, even when the key is loaded, which is also what encrypted file backup/restore (LU-16374) and EC parity resync need; with that, migrate/mirror would copy ciphertext and no longer re-encrypt. sanity-sec test_52 should also be extended to use a multi-MiB file.