Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20462 Lustre Find Utility (LFU)
  3. LU-20602

MDT-internal objects carry no LMA flag marking them internal

XMLWordPrintable

    • Icon: Technical task Technical task
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      A scan that enumerates objects without namespace context classifies each one from `trusted.lma` and its FID, using the ladder in `osd_scrub_get_fid()` /`osd_iit_iget()`. Three MDT-internal objects pass it and come out user-visible:

      Object FID `lma_compat` `lma_incompat`
      `/CONFIGS/mountdata` `[0xe:0x0:0x0]` — IGIF, seq = its own inode number 0 0
      `/update_log_dir/[0x200000400:0x1:0x0]` normal sequence 0 0
      `/update_log_dir/[0x200000401:0x1:0x0]` normal sequence 0 0

      `LMAC_NOT_IN_OI` is set on the OI and sequence directories (`osd_compat.c`,`osd_oi.c`) but not on objects created through `local_storage.c`, nor on`mountdata`; and `fid_is_namespace_visible()` accepts both an IGIF and a normal sequence, so FID range does not separate them either. `FID_SEQ_UPDATE_LOG` / `FID_SEQ_UPDATE_LOG_DIR` would classify the update logs correctly, but the objects observed are not on those sequences — whether that is expected is a question for the reviewer.

      The flag is not what keeps the other internal objects out. No object stamped by `osd_ios_scan_one()` is flagged at all: `last_rcvd`, `CATALOGS`, `reply_data`, `fld`, `seq_ctl`, `seq_srv`, `lov_objid` and `health_check` all take `compat = 0` from `osd_scrub.c:1918`, exactly as a regular file under `ROOT` does. They stay out of a scan only because their `olm_fid` sits on `FID_SEQ_LOCAL_FILE`, which `fid_is_namespace_visible()` rejects. So the MDT's internal objects are held back by two unrelated mechanisms, and the three above are where both miss:

      mechanism covers misses
      `LMAC_NOT_IN_OI` OI files, seq directories, `O/`, index backup dir, fs root everything reached through `osd_ios_scan_one()`
      FID sequence the `FID_SEQ_LOCAL_FILE` objects listed above `mountdata` (IGIF), the two `update_log_dir` children (`FID_SEQ_NORMAL`)

      `/CONFIGS/mountdata` is reached by `osd_ios_varfid_fill()`, which passes `fid = NULL`, so a missing LMA becomes an IGIF built from the inode number – namespace-visible. `update_log_dir` is reached by `osd_ios_uld_fill()`, which parses the FID out of the file name; the two observed are on `0x200000400`/`0x200000401`, the first two normal sequences (`FID_SEQ_NORMAL = 0x200000400`), so they are namespace-visible too.

      Observed on RHEL 9.7, Lustre 2.17, `testfs-MDT0000` on `/dev/vdb`, 1024-byte inodes, MDT mounted and serving: inodes read straight from the inode table, FID set compared against `lfs find` + `lfs path2fid`. Exactly these three appear as extras at both 173 and 509 objects — zero missing, and the leak is bounded rather than proportional. (Three further extras are `.lustre`, `.lustre/fid` and `.lustre/lost+found`, which are genuinely visible and merely hidden from `lfs find`.)

      Impact. Every out-of-namespace scanner needs a hand-maintained denylist of internal inode numbers, tracking every future layout change, in every consumer. A flag on the object is durable; a denylist is not.

      Proposed fix. Add a new compat flag – `LMAC_INTERNAL`, "not part of the user namespace" – and set it in `osd_ios_scan_one()`. Not `LMAC_NOT_IN_OI`: that flag also means "does not need an OI mapping" and is acted on by `osd_scrub_get_fid()`, `osd_oi_insert()` and `osd_dirent_check_repair()`, and the `update_log_dir` children are located by FID and do need their OI mapping maintained.

      Both branches of `osd_ios_scan_one()` need it, not only `osd_scrub.c:1918`. That line runs only when the object has no LMA yet, so on a filesystem that has already completed its initial OI scrub `mountdata` carries an LMA with `lma_compat == 0` and takes the `else` at 1923, where nothing rewrites it. Setting the flag at 1918 alone would produce it on freshly formatted MDTs and never on deployed ones – a consumer would be correct on new filesystems and silently wrong everywhere else. The `else` path must upgrade an LMA that lacks the flag, which makes this an upgrade action: until a filesystem has had one scrub pass with the new code, an unflagged object proves nothing, and consumers need to say so.

            hnishida Hiroshi Nishida
            hnishida Hiroshi Nishida
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: