Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10530

Scanning tool for HSM storage of POSIX copytool

    XMLWordPrintable

Details

    • New Feature
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • 9223372036854775807

    Description

      The current HSM POSIX copytool works well as a demo of the copytool implementation. However, the copytool will create a lot of directoy entries under the HSM storage, which will never be cleaned up by the copytool. Also, if Remove Archive on Last Unlink policy (LU-4640) is not enabled and policy engine like Robinhood is not used, then a lot of orphan files will be left there in HSM storage. Thus, we want to develop a tool which will scan the directory tree of the HSM storage and cleanup the directory three.

      Also, since PCC (LU-10092, LU-10499) also uses the HSM directory structure, this tool could aslo be used for PCC to:

      1) scan the whole PCC storage to detach all the cached files when it is not suitable any more to cache the files on PCC, e.g. when a job stops and another job is starting to use PCC.

      2) apply some kind of policy (e.g. based on UID/GID/JOBID/ProjID, etc) for PCC cache management.

      These use cases needs full scanning of the HSM storage directory, and that is what we want to implement.

      Attachments

        Issue Links

          Activity

            People

              lixi_wc Li Xi
              lixi Li Xi (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: