[LU-19598] TCU: Clean up files from the Trash Can

• Type: Technical task
• Resolution: Unresolved
• Priority: Medium
• Severity: 3
• Assignee: Emoly Liu
• Reporter: Qian Yingjin

Description

A mechanism is needed to automatically clean up files from the trash can when the filesystem becomes full or the files in Trash Can are too old to expire the retain period.

• For emergency case, potentially via MDS kernel thread to skip moving files into Trash Can (destroy the file directly), or delete the oldest pFID tree;
• Userland policy engine can implement more sophisticated policy(age/space by user/projid, or file name, etc). It monitors the space/inode usage on the MDTs and OSTs periodically, once found the space is nearly full:
  • Scan MDT or OST devices for files in Trash Can via policy engine
  • Check TRASH and NOTRASH attributes on file (bot MDT and OST)
  • Check the original UID/GID/PROJID and deletion timestamp in "trusted.unrm" XATTR
  • Choose the candidate files or directories to be purged from Trash Can to free up space;

To improve the performance, we can use multiple threads to do file purge or improve the mpiFileUtils to scan and purge the files in Trash Can in parallel.