Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-12714 PCC Phase 2
  3. LU-12373

detach and delete the PCC cached files when remove a PCC backend from a client

    Details

    • Type: Technical task
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Lustre 2.14.0
    • Labels:
      None
    • Rank (Obsolete):
      9223372036854775807

      Description

      Currently when remove a PCC backend from a client, it does not deal with previously PCC-cached files at all. These files can still use PCC caching service.

      That's to say, after remove the corresponding PCC backend:

      1) For RW-PCC cached files, we do not restore the data back into Lustre OSTs of the main filesystem. Although the PCC backend falls back as a tranditional HSM storage solution since the lhsmtool_posix copytool is still running at this client. But this is dangerous, and likly to cuase user data to be lost if the PCC device maybe permanently unaviailable.

      2) The space used by these PCC cached files may not release.

       

      Thus, we need to add an option to detach and delete the PCC cached files when remove a PCC backend from a client.

      The practical and feasible methods is to scan the PCC backend fs, execute the detach operations via "lfs pcc detach" command or the corresponding llapi.

      i.e.  For RW-PCC cached files, it can do as follows:

       "find /mnt/pcc -type f -name '[0-9a-f:]'" to generate a FID list, then "lfs pcc detach_fid $fid" for each FID.

       

      Here we still need to patch the PCC detach, as the the scanned regular files in the PCC backend fs may be detached involuntary by the Layout lock revocation due to the LRU lock shrinking or by the inode reclaim. 

      For these kind of files,  PCC detach still needs to restore the file into Lustre OSTs, wait the restore finish, and then send HSM remove requests to remove them for RW-PCC; while for RO-PCC, we need to simply delete the crresponding files.

       

       

        Attachments

          Activity

            People

            • Assignee:
              qian_wc Qian Yingjin
              Reporter:
              qian_wc Qian Yingjin
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: