Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10938

Metadata writeback cache support

    XMLWordPrintable

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • 9223372036854775807

    Description

      Metadata writeback cache is a highly desirable feature that should allow Lustre to overcome link latency for modifying metadata operations esp. noticeable in interactive type of workloads.

      Examples include upacking large archives and building software on a single node with no contention from any other nodes.

      Another highly visible set of usecases would include "File/directory-per-process" kind of benchmarks like say mdtest.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-16361 Lustre client was crashed when I used WBC

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. LU-14139 batched statahead processing

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Technical task - A technical task. LU-13015 WBC: lfs wbc and lctl wbc utils

          • Blocker - Blocks development and/or testing work, production could not run.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-11018 async close RPC

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-14138 Move more members in PTLRPC request into pill

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Technical task - A technical task. LU-13008 WBC: memory limits for caching

          • Blocker - Blocks development and/or testing work, production could not run.
          • Open

          Technical task - A technical task. LU-13009 WBC: fsync() support

          • Blocker - Blocks development and/or testing work, production could not run.
          • Open

          Technical task - A technical task. LU-13010 WBC: Reopen the file when WBC EX lock revoking

          • Blocker - Blocks development and/or testing work, production could not run.
          • Open

          Technical task - A technical task. LU-13015 WBC: lfs wbc and lctl wbc utils

          • Blocker - Blocks development and/or testing work, production could not run.
          • Open

          Technical task - A technical task. LU-13007 WBC: Add cache aging to flush dirty cache in the background

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Technical task - A technical task. LU-13016 WBC1.5: Cached data reintegration after the client was evicted

          • Major - Major loss of function.
          • Open

          Technical task - A technical task. LU-13011 WBC2: Integrate PCC with Metadata Writeback Caching

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Technical task - A technical task. LU-13012 WBC: Rule based auto WBC

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          1.
          		 WBC: Add cache aging to flush dirty cache in the background Technical task Open Qian Yingjin
          2.
          		 WBC: memory limits for caching Technical task Open Qian Yingjin
          3.
          		 WBC: fsync() support Technical task Open Qian Yingjin
          4.
          		 WBC: Reopen the file when WBC EX lock revoking Technical task Open WC Triage
          5.
          		 WBC2: Integrate PCC with Metadata Writeback Caching Technical task Open Qian Yingjin
          6.
          		 WBC: Rule based auto WBC Technical task Open Qian Yingjin
          7.
          		 WBC: lfs wbc and lctl wbc utils Technical task Open Qian Yingjin
          8.
          		 WBC1.5: Cached data reintegration after the client was evicted Technical task Open WC Triage
          9.
          		 WBC：different flush modes Technical task In Progress Qian Yingjin
          10.
          		 WBC3: Disconnected operation support Technical task Open Qian Yingjin
          11.
          		 WBC3: remove the whole subtree on MDT already deleted in the client WBC cache Technical task Open WC Triage
          12.
          		 WBC2: batch metadata update Technical task Open Qian Yingjin
          13.
          		 WBC2: lockless IO Technical task Open Qian Yingjin
          14.
          		 WBC: Basic framework for WBC Technical task In Progress Qian Yingjin
          15.
          		 WBC: Add symlink support Technical task Open Qian Yingjin
          16.
          		 WBC: Add hardlink support Technical task Open Qian Yingjin
          17.
          		 WBC: special readdir() handling for root WBC directory Technical task Open Qian Yingjin
          18.
          		 WBC: Reclaim mechanism for cached inodes and pages under limits in MemFS Technical task Open Qian Yingjin
          19.
          		 WBC: inode and space grant mechanism under WBC Technical task Open WC Triage
          20.
          		 WBC: implement mkdir() by using intent lock Technical task Open Qian Yingjin
          21.
          		 WBC: better DNE support Technical task Open Qian Yingjin
          22.
          		 WBC: Stripe directory not flushed to MDT when it grows large enough Technical task Open Qian Yingjin
          23.
          		 WBC: MemFS lookup fallback to Lustre lookup Technical task Open Qian Yingjin
          24.
          		 WBC: trigger flush on background when exceed the dirty inode threshold Technical task Open Qian Yingjin
          25.
          		 WBC: Handle WB_SYNC_NONE properly Technical task Open Qian Yingjin
          26.
          		 WBC: endless loop in balance_dirty_pages Technical task Open Qian Yingjin
          27.
          		 WBC2: it should not cause rename() failed when WBC is disabled Technical task Open Qian Yingjin
          28.
          		 WBC: Write hang with O_DSYNC file flag Technical task Open Qian Yingjin
          29.
          		 WBC: deadlock for buffered write in parallel when test with dcp/ior Technical task Open Qian Yingjin
          30.
          		 WBC: Recovery mechanism for batched RPC Technical task Open WC Triage
          31.
          		 WBC: replay recovery and layout instantiation Technical task Open WC Triage
          32.
          		 WBC: rename() support for WBC Technical task Open Qian Yingjin
          33.
          		 WBC: uncache a given file or directory from WBC Technical task Open Qian Yingjin
          34.
          		 WBC: Handling writeback errors and retry for recoverable errors Technical task Open WC Triage
          35.
          		 WBC: discard the cached subtrees under WBC when the client is evicted Technical task Open Qian Yingjin

          Activity

            People

              qian_wc Qian Yingjin
              green Oleg Drokin
              Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated: