Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10092

PCC: Lustre Persistent Client Cache

    XMLWordPrintable

Details

    • New Feature
    • Resolution: Fixed
    • Minor
    • Lustre 2.13.0
    • None
    • 9223372036854775807

    Description

      PCC is a new framework which provides a group of local cache on Lustre client side. No global namespace will be provided by PCC. Each client uses its own SSD as a local cache for it self. Local file system is used on the SSD to manage the data on local caches. Cached I/O is directed to local file system while normal I/O is directed to OSTs.

      PCC uses HSM for data synchronization. It uses HSM copytool to restore file from local caches to Lustre OSTs. Each PCC has a copytool instance running with unique archive number. Any remote access from another Lustre client would trigger the data synchronization. If a client with PCC goes offline, the cached data becomes inaccessible for other client temporally. And after the PCC client reboots and the copytool restarts, the data will be accessible again.

      Following is what will happen in different conditions:

      When file is being created on PCC

      • A normal file is created on MDT;
      • An empty mirror file is created on local cache;
      • The HSM status of the Lustre file will be set to archived and released;
      • The archive number will be set to the proper value.

      When file is being **prefetched* to PCC*

      • An mirror file is copied to local cache;
      • The HSM status of the Lustre file will be set to archived and released;
      • The archive number will be set to the proper value.

      When file is being accessed from PCC

      • Data will be read directly from local cache;
      • Metadata will be read from MDT, except file size;
      • File size will be got from local cache.

      PCC should be able to accelerate some applications with certain I/O patterns.

      For more information, please check the presentation during LUG'18 (http://wiki.lustre.org/images/0/04/LUG2018-Lustre_Persistent_Client_Cache-Xi.pdf).

      Attachments

        Issue Links

          Activity

            People

              lixi_wc Li Xi
              lixi Li Xi (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              29 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: