Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10938

Metadata writeback cache support

XMLWordPrintable

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • None
    • 9223372036854775807

      Metadata writeback cache is a highly desirable feature that should allow Lustre to overcome link latency for modifying metadata operations esp. noticeable in interactive type of workloads.

      Examples include upacking large archives and building software on a single node with no contention from any other nodes.

      Another highly visible set of usecases would include "File/directory-per-process" kind of benchmarks like say mdtest.

        1. LUG2018-Developer_Day-Client_Metadata_Writeback_Cache-Oleg_Drokin.pdf
          464 kB
        2. MetaWBC_updated.pptx
          6.07 MB
        3. WBC.pptx
          1.12 MB
        1.
        		 WBC: Add cache aging to flush dirty cache in the background Technical task Open Qian Yingjin
        2.
        		 WBC: memory limits for caching Technical task Open Qian Yingjin
        3.
        		 WBC: fsync() support Technical task Open Qian Yingjin
        4.
        		 WBC: Reopen the file when WBC EX lock revoking Technical task Open WC Triage
        5.
        		 WBC2: Integrate PCC with Metadata Writeback Caching Technical task Open Qian Yingjin
        6.
        		 WBC: Rule based auto WBC Technical task Open Qian Yingjin
        7.
        		 WBC: lfs wbc and lctl wbc utils Technical task Open Qian Yingjin
        8.
        		 WBC1.5: Cached data reintegration after the client was evicted Technical task Open WC Triage
        9.
        		 WBC：different flush modes Technical task In Progress Qian Yingjin
        10.
        		 WBC3: Disconnected operation support Technical task Open Qian Yingjin
        11.
        		 WBC3: remove the whole subtree on MDT already deleted in the client WBC cache Technical task Open WC Triage
        12.
        		 WBC2: batch metadata update Technical task Open Qian Yingjin
        13.
        		 WBC2: lockless IO Technical task Open Qian Yingjin
        14.
        		 WBC: Basic framework for WBC Technical task In Progress Qian Yingjin
        15.
        		 WBC: Add symlink support Technical task Open Qian Yingjin
        16.
        		 WBC: Add hardlink support Technical task Open Qian Yingjin
        17.
        		 WBC: special readdir() handling for root WBC directory Technical task Open Qian Yingjin
        18.
        		 WBC: Reclaim mechanism for cached inodes and pages under limits in MemFS Technical task Open Qian Yingjin
        19.
        		 WBC: inode and space grant mechanism under WBC Technical task Open WC Triage
        20.
        		 WBC: implement mkdir() by using intent lock Technical task Open Qian Yingjin
        21.
        		 WBC: better DNE support Technical task Open Qian Yingjin
        22.
        		 WBC: Stripe directory not flushed to MDT when it grows large enough Technical task Open Qian Yingjin
        23.
        		 WBC: MemFS lookup fallback to Lustre lookup Technical task Open Qian Yingjin
        24.
        		 WBC: trigger flush on background when exceed the dirty inode threshold Technical task Open Qian Yingjin
        25.
        		 WBC: Handle WB_SYNC_NONE properly Technical task Open Qian Yingjin
        26.
        		 WBC: endless loop in balance_dirty_pages Technical task Open Qian Yingjin
        27.
        		 WBC2: it should not cause rename() failed when WBC is disabled Technical task Open Qian Yingjin
        28.
        		 WBC: Write hang with O_DSYNC file flag Technical task Open Qian Yingjin
        29.
        		 WBC: deadlock for buffered write in parallel when test with dcp/ior Technical task Open Qian Yingjin
        30.
        		 WBC: Recovery mechanism for batched RPC Technical task Open WC Triage
        31.
        		 WBC: replay recovery and layout instantiation Technical task Open WC Triage
        32.
        		 WBC: rename() support for WBC Technical task Open Qian Yingjin
        33.
        		 WBC: uncache a given file or directory from WBC Technical task Open Qian Yingjin
        34.
        		 WBC: Handling writeback errors and retry for recoverable errors Technical task Open WC Triage
        35.
        		 WBC: discard the cached subtrees under WBC when the client is evicted Technical task Open Qian Yingjin

            qian_wc Qian Yingjin
            green Oleg Drokin
            Votes:
            0 Vote for this issue
            Watchers:
            23 Start watching this issue

              Created:
              Updated: