[LU-12359] Remote shared burst buffer PCC on a shared backend fs

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Assignee: Qian Yingjin
• Reporter: Qian Yingjin

Description

HPC burst buffers are a fast storage layer positioned between the compute engines and the backend storage systems.

There are two representative burst buffer architectures: remote shared burst buffers and node-local burst buffer. DataWarp and Infinite Memory Engine belong to the former. In the case of remote shared burst buffers, the SSD storage resides in I/O nodes positioned between the compute nodes and the backend storage. Data movement between compute nodes and the burst buffer needs to go through a network. Placing burst buffers in I/O nodes facilitates their independent development, deployment, and maintenance. The aggregate bandwidth of node local burst buffers grows linearly with the number of compute nodes. Node-local burst buffers
also require a scalable metadata management to maintain a global namespace across all nodes.

RW-PCC provides an elegant way to couple node-local burst buffers with Lustre. The metadata is managed by Lustre and stored on MDTs. Thus, it becomes part of the global Lustre namespace. Moreover, the file data can be migrated from the LPCC cache to the Lustre OSTs via file restores, and it is transparent to the application. Furthermore,we can customize various cache strategies and provide cache isolation according to files’ attributes.

 

Although the node-local PCC nearly does not occupy any network resrouce when perform data IO, but the capacity of the node-local PCC is limited by the storage media on this client.

 

A novel remote shread PCC for Lustre filesystem is proposed, which can be used as a remote shared burst buffer on a shared PCC backend fs. This shared PCC backend fs could be a high speed networked filesystem (i.e. another Lustre storage) using high speed NVMe or SSD while the current Lustre filesystem is minaly using slow speed HDDs. 

By this way, all Lustre clients can use the shared PCC backend fs with larger capacity. And we can have 4 level storage tires for a single Lustre filesystem

• OST storage tire
• Original node-local PCC
• Remote shared PCC on a shared backend fs
• Traditional Lustre HSM solution

 

The implementation of remote shared PCC can use the Foundation and framework of current node-local PCC.

Moreover, Under the remote shared RO-PCC, once a file is attached into the shared PCC backend fs, it can sharely read from PCC by all clients.

For the remote shared RW-PCC, it works as original, and can only read/write by a single client.

Issue Links

• is related to: LU-10606 HSM info as part of LOV layout xattr