Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20462 Lustre Find Utility (LFU)
  3. LU-20887

LFU: Object Stream of an in-service OST does not carry the owner FID

XMLWordPrintable

    • Icon: Technical task Technical task
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      Problem

      An OST object records the MDT file it belongs to: the parent FID in the object's trusted.fid, or in the LMA on OSTs with 256-byte inodes (LMAC_STRIPE_INFO).

      LFU already reports it when it reads a target directly. For a stopped OST, llapi_scan_device() answers LLAPI_SCAN_OWNER / lfsr_owner_fid (LU-20606, LU-20637), and --fid2path names each object by its file.

      For an OST that is mounted and serving, the scan goes through the Object Stream from the OSD otable iterator (LU-20720, LU-20722). Its record, struct lfu_rec, carries the attributes, trusted.lma, trusted.som, trusted.link and the presence of LOV/LMV, but not trusted.fid. So a scan of an in-service OST lists objects but cannot say which files they belong to.

      Why it matters

      This is the scan an OST drain needs: the OST stays mounted while its files are migrated away, and the list of files with objects on it should come from that one OST instead of a scan of every MDT. The same applies to shrinking an OST and to a parallel data mover that consumes the LFU stream.

      Reading the mounted device from outside is not an alternative, because it gives stale results. In a test, a scan of the raw disk of a mounted ldiskfs OST named none of the newly written files, and debugfs on a live OST found 441 of 1073 files.

      Proposed fix

      A patch on top of the LU-20720 / LU-20722 / LU-20730 series:

      • dt_otable_lfu_rec() reads the parent FID through DORA_XATTR, from trusted.fid or from the LMA, only for OST objects (and objects with no LMA), so MDT scans do not pay for it.
      • The FID is carried in the record's tail after the link, as struct lfu_owner, flagged by a new LFU_REC_HAVE_OWNER bit. The header size and LFU_WIRE_VERSION do not change, so older readers skip it and the client OBD_IDX_READ stream stays compatible.
      • Userspace hands it to the scanner as the head of a trusted.fid, so LLAPI_SCAN_OWNER and --fid2path work on a mounted OST as on a stopped one.
      • New conf-sanity test_305.

      Tested on el9 (kernel 5.14, ldiskfs): test_305 passes with the patch (20 written files named by owner, never-written objects not named) and fails with the kernel-side change reverted.

      Open points

      • Objects that were created but never written or setattr'ed have no parent FID on the OST, so they are still reported without an owner. For a drain they need either the parent FID written at create time or a layout-only check on the MDT.
      • Not tested: ZFS OSTs, OSTs with 256-byte inodes (the LMA path), the client OBD_IDX_READ stream.
      • Passing the raw disk of a target that is mounted through a mapped device (for example dm-flakey) silently falls back to reading the live disk from outside. The scan should refuse or warn.

            ablagodarenko Artem Blagodarenko
            ablagodarenko Artem Blagodarenko
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: