Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20934

Client ".." (get_parent) lookup fails with ENOENT/ESTALE via stale master FID after striped directory migration

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • Lustre 2.16.0
    • 3
    • 9223372036854775807

      Origin: IME/DDN ticket EX-16006. This is a common issue on all branches; fix on master first, then backport. h3. Problem In a DNE filesystem, when a striped directory is migrated between MDTs (lfs migrate -m <mdt> <dir>), the directory's master object is replaced (new master FID, old master object destroyed). A client that already has the old master FID cached in its LMV layout (its inode was revalidated before the migration) keeps using that stale FID. When the VFS resolves .. through that directory (parent lookup during path walk, NFS get_parent, ls dir/../file), the MDT returns -ENOENT/-ESTALE, and the whole path resolution fails on the client until its cached layout is refreshed — which it never gets, because the lookup itself is what would refresh it. This is reproducible with two clients: client A migrates the directory while client B holds the stale master FID. h3. Reproducer (4-MDT DNE, >= 2 clients)

       client A: lfs mkdir -i 0 -c 4 /mnt/lustre/d0/sub1     # striped dir on MDT0 client B: stat /mnt/lustre/d0/sub1            # caches master FID, e.g. 0x200000403:0x1 client A: lfs migrate -m 1 /mnt/lustre/d0/sub1 # master FID changes client B: ls /mnt/lustre/d0/sub1/../f1         # -ENOENT / -ESTALE 

      h3. Root cause lmv_getattr_name() in lustre/lmv/lmv_obd.c resolves .. by sending MDS_GETATTR_NAME to the MDT hosting op_fid1 — the (possibly stale) master FID. After the migration completes on another client, that master object no longer exists, and there is no recovery path: the existing retry logic only handles the in-progress-migration case (lmv_dir_retry_check_update()), not the "migration already completed by another client" case. h3. Fix (client-side, no server change) In lmv_getattr_name(), when a .. lookup on a striped directory returns -ENOENT/-ESTALE, retry once with op_fid1 set to the first stripe's FID (lsm_md_oinfo[0].lmo_fid), guarded by a dir_stale flag so it happens at most once. The stripes survive the migration and their .. entries are re-pointed at the new master, so the server's mdt_raw_lookup() returns the new master FID, which flows back through ll_prep_inode() -> ll_update_lsm_md() and permanently heals the client's cached layout. h3. Test New sanity.sh test_230D: two clients; client2 warms the subdir master FID; the subdir is migrated across MDTs twice; client2 drops caches and MDC locks and resolves .. through the stale FID after each migration. PASS with the fix on a 3-node 4-MDT cluster; no regressions in sanity 230l/230w/230y/230z.

            laisiyao Lai Siyao
            laisiyao Lai Siyao
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: