Metadata writeback cache support
(LU-10938)
|
|
| Status: | Open |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Technical task | Priority: | Minor |
| Reporter: | Qian Yingjin | Assignee: | Qian Yingjin |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||
| Description |
|
In current WBC prototype, all newly directories are created with subtree EX lock returned (WBC). All directories are treated equally with metadata writeback caching. There is no any customization. It can define various auto caching rule for WBC when create a new directory similar to TBF or PCC. When a new create directory meets the rule condition, it can try to obtain EX WBC lock from MDS and keep exclusive access on the directory under the protection of the EX lock on the client. The rule can be combination of uid/gid/projid/fname or jobid.
# lctl wbc $MOUNT $rule
# lctl wbc /mnt/lustre “projid={100}”
# lctl wbc /mnt/lustre “uid={500}&gid={500},fname={*dirmatch*}”
# lctl wbc /mnt/lustre “jobid={dd.*}”
|
| Comments |
| Comment by Andreas Dilger [ 27/Nov/19 ] |
|
I suspect that in most cases this should not be needed. Caching metadata does not take a lot of space compared to file data, and if the background age and lock cancellation is working efficiently then WBC should be totally transparent to the user. The main use for this would be to allow tuning the WBC state of existing directories if a simple policy like "ask MDS for EX lock after N creates in existing directory" is not enough. The MDS can always refuse the EX lock and just create the file if there is contention in that directory. This should be considered a WBC v2 "nice to have" goal, but I don't think it is needed for v1. It would probably make more sense to spend time to optimize the writeback performance (eg. batch RPCs) so that cache flush can happen quickly and efficiently in the background. |
| Comment by Gerrit Updater [ 12/May/22 ] |
|
"Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47312 |
| Comment by Gerrit Updater [ 13/May/22 ] |
|
"Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47338 |
| Comment by Gerrit Updater [ 17/May/22 ] |
|
"Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47367 |
[LU-13012]