Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20462 Lustre Find Utility (LFU)
  3. LU-20606

LFU: implement ldiskfs device scanner

XMLWordPrintable

    • Icon: Technical task Technical task
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • Lustre 2.18.0
    • None
    • 3
    • 9223372036854775807

      Implement the ldiskfs block device scanner using libext2fs to achieve high-performance MDT and OST object scanning.

      Generate an LFU Object Stream by reading MDT (and OST) metadata directly from the block device in userspace, via libext2fs, at device read bandwidth. This replaces client-side lfs find traversal for the initial rollout.

      It follows e2scan (in Lustre e2fsprogs history) and Lester, the Lustre Lister. It runs independently of the MDS/OSS service and needs only read-only access to the underlying block device.

      In scope

      • Full-device enumeration of ldiskfs inodes on an MDT or OST
      • FID recovery and object classification from Lustre xattrs
      • Attribute extraction into an Object Stream, driven by a caller-supplied mask
      • Pushed-down filter evaluation, ordered by I/O cost
      • Parallel scan within one device; checkpoint and restart

      Out of scope

      • Repair of any kind. LFU reads; LFSCK and OI Scrub repair. The scanner has no write path and opens the device O_RDONLY.
      • Pathname construction — an Output Format concern (Path resolution priced separately), though this module emits the linkea bytes that make it possible.
      • Cross-target merge — a Filter Rule concern, server-side
      • ZFS and WBCFS backends — those are served by the OSD API Input Scanner, which is why that module exists.

      Why this module first

      It requires no kernel changes and no Lustre server modification, so it works against unmodified existing servers. That is what makes the "no flag day" rollout real.

            hnishida Hiroshi Nishida
            adilger Andreas Dilger
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: