[LU-10499] Readonly Persistent Client Cache support

• Type: Technical task
• Resolution: Unresolved
• Priority: Minor
• Assignee: Qian Yingjin
• Reporter: Li Xi

Description

Readwrite Persistent Client Cache has been implemented by using HSM features (LU-10092). When a file is cached on Readwrite PCC (RW-PCC), only this single client can read/write to the file. RW-PCC can accelerate I/O intensive applications which read/write data into a file in a single client. However, in some other use cases, part of the files will never be updated, but need to be readed at the same time from multiple clients. Readonly Persistent Client Cache tries to implement a readonly cache level on Lustre clients using SSDs/NVMe.

Readonly Persistent Client Cache (RO-PCC) shares the same framework with Readwrite Persistent Client Cache, except that no HSM mechanism is used in readonly mode of PCC. Instead, readonly mode of PCC uses grouplock to protect the data from being written from other clients.

Whenever a file is fetched into RO-PCC, like the archive number of RW-PCC, a grouplock ID needs to be specified. The grouplock of this ID will be held in the client as long as the file is being cached in RO-PCC. Multiple clients can fetch the same file into its local cache using the same grouplock ID. Because of the semantics of grouplock, no other client can read/write from/to the file if the grouplock with the same ID is not held. Also because of the semantics of grouplock, no client can fetch the file to RO-PCC with a different grouplock ID.

Issue Links

• is blocking: LU-10092 PCC: Lustre Persistent Client Cache
• is related to: LU-13602 Skip unknown FLR component types
• is related to: LU-13842 Can't add pcc cache with roid only parameter.
• is related to: LU-13843 removing non existing path with lctl pcc del returns 0 (success) exit code without error message
• is related to: LU-13841 error messages need additional explanation for lctl pcc commands
• is related to: LU-10602 Add file heat support for Persistent Client Cache
• is related to: LU-10606 HSM info as part of LOV layout xattr
• is related to: LU-13889 mechanism for PCC to get new archive ID
• is related to: LU-13924 Document PCC "lfs pcc add" options
• is related to: LU-16700 reserve flags and define data structure for PCC-RO

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54463/
Subject: LU-10499 pcc: tunable GID to pin/unpin files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 84100d7f7d38cd7d7c612c190359d27a0554e011

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54461/
Subject: LU-10499 pcc: avoid to specify attach ID for every pin
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: abc39d438bff3d16618b22b5c4da312c2fc29892

[Gerrit Updater]

"James Simmons <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57671
Subject: LU-10499 pcc: Add --pin option for 'lfs pcc attach' command
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 977db583ac3d36cfcf53c22b9a89fdcc7139355d

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54460/
Subject: LU-10499 utils: add pcc pin/unpin commands
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: aae909cf8fbc5ef3090c5385793370fa71360f42

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54459/
Subject: LU-10499 sec: support of PCC-RO for encrypted files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 9e37963fa05b66652610771740512971c65a47ef

[Gerrit Updater]

"James Simmons <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57617
Subject: LU-10499 utils: use libyaml for pin xattr handling
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: fc2cbaacf0d3580cb313ebaf59a0354e4cc4902f

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54486/
Subject: LU-10499 tests: disable metadata_csum_seed for pcc cache device
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 97fc40f62aa03e17ebde9a698f33ed58cc2e477e

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54458/
Subject: LU-10499 pcc: add some statistics data
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 34903b22e51422db14828f898974cf0d1e81b56e

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54457/
Subject: LU-10499 pcc: command to remove PCC mirror component
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 01b82baffed95d64fd564229c829543a0e57bf31

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54451/
Subject: LU-10499 pcc: Check if PCC copy is unlinked for state output
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3fcc8bc896c548e7651bf279041b5fd6234c4989