Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20462 Lustre Find Utility (LFU)
  3. LU-21151

LFU: lfu-purge, a daemon that frees space in Fast Tier OST and PCC-RO caches

XMLWordPrintable

    • Icon: Technical task Technical task
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      Fast Tier OSTs and PCC-RO caches hold copies of data that can be dropped when
      space runs low. Nothing reclaims them by policy today.

      Add lfu-purge, as described in the lfu-purge HLD by Andreas:

      • One process per cache keeps free space and free inodes between free_min and free_max.
      • A scanner thread finds candidates with LFU scans: llapi_scan_device() on a Fast Tier OST, llapi_scan_namespace() on a PCC-RO dataset. It keeps them in a weighted-age histogram, so the main thread can purge from the old end without waiting for a scan.
      • The scan rate follows free space.
      • Policy classes use lfs find predicates, with weights and used_min/used_max.
      • Before a purge, each candidate is checked again: its age, pins, opens, and for a fast mirror that another mirror is in sync.
      • For a striped component, the OST with the first stripe owns the purge.

      Also add llapi_mirror_split(): the daemon must drop a mirror as lfs mirror split -d does, and that code is static in lfs.c now.

      Phase 1 of the HLD: on an OSS, the daemon needs a client mount. Setting up PCC-RO from mount -o pcc waits for LU-21082.

            hnishida Hiroshi Nishida
            hnishida Hiroshi Nishida
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: