Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-21177

volatile create in .lustre/fid prunes all cached FID dentries under the directory's d_lock

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • Upstream, Lustre 2.17.0
    • None
    • 3
    • 9223372036854775807

      Tools that move files by FID (lfs migrate, lfs mirror extend/split, movers and policy engines through llapi_create_volatile_idx(), the HSM copytool restore) create their volatile files in <mnt>/.lustre/fid. When the file goes to MDT0, every create cancels the client's UPDATE lock on .lustre/fid, and the cancel runs ll_prune_negative_children(), which walks one cached dentry per FID ever looked up under the d_lock of .lustre/fid. All FID path walks on the client spin on that lock meanwhile.

      The cancel is not needed: a volatile file gets no name in the parent, and since LU-14693 (2.14.52) the MDT takes no lock on the parent for it. It happens twice per create:

      1. mdc_intent_open_pack() cancels the parent's UPDATE lock by ELC for every create, volatile or not.
      2. liblustreapi's create_volatile() and create_restore_volatile() unlink() the volatile name afterwards (for non-Lustre file systems). On Lustre that is an RPC failing with -ENOENT, and mdc_unlink() cancels the same lock first.

      One el9.8 client, master servers, 400k cached FIDs in .lustre/fid, volatile file on MDT0:

      client master mdc patch mdc + llapi patches
      llapi_create_volatile_idx() in .lustre/fid 90/s 156/s 1220/s
      prunes per create, each 2, 4.4 ms 1, 4.8 ms 0
      raw volatile open() as lfs does it 174/s 1234/s 1227/s
      6 threads stat-ing FIDs beside llapi creates (84,000/s alone) 3,134/s 7,812/s 78,319/s

      Fix, two patches: no parent ELC in mdc_intent_open_pack() for MDS_CREATE_VOLATILE (an older MDT revokes the lock by blocking callback instead), and no unlink() in liblustreapi when the volatile open succeeded on Lustre, with sanity test_185b.

            wc-triage WC Triage
            markus.hilger Markus Hilger
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: