Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10530

Scanning tool for HSM storage of POSIX copytool

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None
    • Rank (Obsolete):
      9223372036854775807

      Description

      The current HSM POSIX copytool works well as a demo of the copytool implementation. However, the copytool will create a lot of directoy entries under the HSM storage, which will never be cleaned up by the copytool. Also, if Remove Archive on Last Unlink policy (LU-4640) is not enabled and policy engine like Robinhood is not used, then a lot of orphan files will be left there in HSM storage. Thus, we want to develop a tool which will scan the directory tree of the HSM storage and cleanup the directory three.

      Also, since PCC (LU-10092, LU-10499) also uses the HSM directory structure, this tool could aslo be used for PCC to:

      1) scan the whole PCC storage to detach all the cached files when it is not suitable any more to cache the files on PCC, e.g. when a job stops and another job is starting to use PCC.

      2) apply some kind of policy (e.g. based on UID/GID/JOBID/ProjID, etc) for PCC cache management.

      These use cases needs full scanning of the HSM storage directory, and that is what we want to implement.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lixi_wc Li Xi
                Reporter:
                lixi Li Xi (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated: