[LU-13012] WBC: Rule based auto WBC

• Type: Technical task
• Resolution: Unresolved
• Priority: Minor
• Assignee: Qian Yingjin
• Reporter: Qian Yingjin

Description

In current WBC prototype,  all newly directories are created with subtree EX lock returned (WBC). All directories are treated equally with metadata writeback caching. There is no any customization.

It can define various auto caching rule for WBC when create a new directory similar to TBF or PCC.

When a new create directory meets the rule condition, it can try to obtain EX WBC lock from MDS and keep exclusive access on the directory under the protection of the EX lock on the client.

The rule can be combination of uid/gid/projid/fname or jobid.

# lctl wbc $MOUNT $rule
# lctl wbc /mnt/lustre “projid={100}”
# lctl wbc /mnt/lustre “uid={500}&gid={500},fname={*dirmatch*}”
# lctl wbc /mnt/lustre “jobid={dd.*}” 

Issue Links

• is related to: LU-10938 Metadata writeback cache support
• is related to: LU-13015 WBC: lfs wbc and lctl wbc utils

Comments

[Gerrit Updater]

"Yingjin Qian <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/47367
Subject: LU-13012 wbc: customizable rule based auto WBC
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2968194daf58c4dbe6e5361dbde33a381e4c986c

[Gerrit Updater]

"Yingjin Qian <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/47338
Subject: LU-13012 pcc: abstract rule parsing and matching func
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: be131170885253952332d7f8083f3e4247fd5354

[Gerrit Updater]

"Yingjin Qian <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/47312
Subject: LU-13012 wbc: auto WBC based on the rule of project ID
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8c18008e01c2636bcd801af945d436394cc7097e

[Andreas Dilger]

I suspect that in most cases this should not be needed. Caching metadata does not take a lot of space compared to file data, and if the background age and lock cancellation is working efficiently then WBC should be totally transparent to the user. The main use for this would be to allow tuning the WBC state of existing directories if a simple policy like "ask MDS for EX lock after N creates in existing directory" is not enough. The MDS can always refuse the EX lock and just create the file if there is contention in that directory.

This should be considered a WBC v2 "nice to have" goal, but I don't think it is needed for v1. It would probably make more sense to spend time to optimize the writeback performance (eg. batch RPCs) so that cache flush can happen quickly and efficiently in the background.