[LU-13021] WBC：different flush modes

• Status: In Progress
• Project: Lustre
• Type: Technical task
• Priority: Minor
• Reporter: Qian Yingjin
• Assignee: Qian Yingjin
• Resolution: Unresolved

Description

To meet different requirements, we design three flush modes for WBC:

• WBC_FLUSH_LAZY
  In the lazy flush mode, a client never flushes dirty cache and drops the root WBC EX lock actively unless it has to in the following cases:
  • The client needs to revoke the cached root WBC EX lock when the directory is Conflict accessing from a remote client or shrink the LRU locks in the client lock namespace;
  • The capacity of the cache on the client is used out;
  • The capacity limits for the cache on the client is reached;
  • A application or a user wants to cleanup or uncache the cached data on the client manually.

This cache mode comes from the following requirements:
– Users want to reduce the impact on the performance: When flushing metadata to MDT and assimilate data, it may block newly creations and data I/O, and have impact on the performance.
– The application may create many files and remove them later. It may do useless work if flush the cache periodically. If the file was flushed to MDT, it may still need to be deleted on MDT later
– Use case:
lfs wbc cache $DIR
do a job under directory $DIR, but do not flush the cache.
lfs wbc uncache $DIR
Sync the cached root WBC directory $DIR, cleanup the corresponding data on cache

• WBC_FLUSH_AGE_LOCK_REVOKE
  In this flush mode, it will trigger to flush dirty cache in the background periodically when the cache is aged.
  The WBC EX locks are dropped level by level during flushing.
  Hold the root WBC EX lock until:
  • Push all its children directories or files to MDT;
  • Acquire the WBC EX lock back on the next level children directories;
    And then release the root WBC EX lock;
• WBC_FLUSH_AGE_LOCK_HOLD
  This flush mode is similar to WBC_FLUSH_AGE_LOCK_REVOKE. Instead, the WBC EX lock for the root WBC directory never drops actively unless it has to in the following cases:
  • The client needs to revoke the cached root WBC EX lock when the directory is Conflict accessing from a remote client or shrink the LRU locks in the client lock namespace;
  • A application or a user wants to cleanup or uncache the cached data on the client manually.

WBC_FLUSH_AGE_LOCK_HOLD should be the default flush mode for WBC. It means that when flushing directories or files under the root WBC directory due to cache aged, it does not need to revoke the root WBC EX lock, acquire WBC EX lock back on its children directories level by level; The files or directories under the root WBC directory can directly flush to MDT (marked with Sync(S) state);

Before the root WBC EX lock is revoked, nearly all operations for the files or directories under the root WBC directory can be executed locally such as open()/close(), getattr(), setattr(), readdir(), read(), write().
But unlink() should be handled specially for files marked with Sync(s) state. In the normal mode, it can perform unlink on MDT directly; while in the disconnected mode, it may need to mark this file deleted, and readdir()/lookup() should ignore this file. After the connection recovered, then unlink the file on MDT.

Or we can do unlink on a already removed directory under the root WBC directory as LU-13044:WBC: remove the whole subtree on MDT already deleted in the client WBC cache

Comments

[Andreas Dilger]

I would think that WBC_FLUSH_LAZY is very unlikely to be used with Lustre. If the application knows that it will create and unlink a whole directory subtree during the job run, and this client is the only one using the subtree, then there is little benefit to putting the files in a shared filesystem. It could instead use tmpfs directly and save the overhead?

[Qian Yingjin]

agree.

However, WBC_FLUSH_LAZY is origin from the base patches of Oleg. Three flush modes are derived from the same origin. It would better to implement them step by step.

And moreover, we may want to compare the performance between WBC with the lazy mode and tmpfs when there is no limits on the memory usage for improvement WBC. Aged flushing may have some impact on the comparison.

[Gerrit Updater]

Yingjin Qian ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37137
Subject: LU-13021 wbc: add aging keep cache flush mode for WBC
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: dd8c00c04b851100aff84b4064216883e59a7c68

[Gerrit Updater]

Yingjin Qian ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37142
Subject: LU-13021 wbc: shrink WBC cached files during umount
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: daa787e64645fab5797a00543429d7148dad2ee1