Details

    • Technical task
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • 9223372036854775807

    Description

      In current WBC prototype,  all newly directories are created with subtree EX lock returned (WBC). All directories are treated equally with metadata writeback caching. There is no any customization.

      It can define various auto caching rule for WBC when create a new directory similar to TBF or PCC.

      When a new create directory meets the rule condition, it can try to obtain EX WBC lock from MDS and keep exclusive access on the directory under the protection of the EX lock on the client.

      The rule can be combination of uid/gid/projid/fname or jobid.

      # lctl wbc $MOUNT $rule
      # lctl wbc /mnt/lustre “projid={100}”
      # lctl wbc /mnt/lustre “uid={500}&gid={500},fname={*dirmatch*}”
      # lctl wbc /mnt/lustre “jobid={dd.*}” 

      Attachments

        Issue Links

          Activity

            [LU-13012] WBC: Rule based auto WBC

            "Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47367
            Subject: LU-13012 wbc: customizable rule based auto WBC
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 2968194daf58c4dbe6e5361dbde33a381e4c986c

            gerrit Gerrit Updater added a comment - "Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47367 Subject: LU-13012 wbc: customizable rule based auto WBC Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 2968194daf58c4dbe6e5361dbde33a381e4c986c

            "Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47338
            Subject: LU-13012 pcc: abstract rule parsing and matching func
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: be131170885253952332d7f8083f3e4247fd5354

            gerrit Gerrit Updater added a comment - "Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47338 Subject: LU-13012 pcc: abstract rule parsing and matching func Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: be131170885253952332d7f8083f3e4247fd5354

            "Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47312
            Subject: LU-13012 wbc: auto WBC based on the rule of project ID
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 8c18008e01c2636bcd801af945d436394cc7097e

            gerrit Gerrit Updater added a comment - "Yingjin Qian <qian@ddn.com>" uploaded a new patch: https://review.whamcloud.com/47312 Subject: LU-13012 wbc: auto WBC based on the rule of project ID Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 8c18008e01c2636bcd801af945d436394cc7097e

            I suspect that in most cases this should not be needed. Caching metadata does not take a lot of space compared to file data, and if the background age and lock cancellation is working efficiently then WBC should be totally transparent to the user. The main use for this would be to allow tuning the WBC state of existing directories if a simple policy like "ask MDS for EX lock after N creates in existing directory" is not enough. The MDS can always refuse the EX lock and just create the file if there is contention in that directory.

            This should be considered a WBC v2 "nice to have" goal, but I don't think it is needed for v1. It would probably make more sense to spend time to optimize the writeback performance (eg. batch RPCs) so that cache flush can happen quickly and efficiently in the background.

            adilger Andreas Dilger added a comment - I suspect that in most cases this should not be needed. Caching metadata does not take a lot of space compared to file data, and if the background age and lock cancellation is working efficiently then WBC should be totally transparent to the user. The main use for this would be to allow tuning the WBC state of existing directories if a simple policy like "ask MDS for EX lock after N creates in existing directory" is not enough. The MDS can always refuse the EX lock and just create the file if there is contention in that directory. This should be considered a WBC v2 "nice to have" goal, but I don't think it is needed for v1. It would probably make more sense to spend time to optimize the writeback performance (eg. batch RPCs) so that cache flush can happen quickly and efficiently in the background.

            People

              qian_wc Qian Yingjin
              qian_wc Qian Yingjin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: