https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10938 Lustre <p>Metadata writeback cache is a highly desirable feature that should allow Lustre to overcome link latency for modifying metadata operations esp. noticeable in interactive type of workloads.</p> <p>Examples include upacking large archives and building software on a single node with no contention from any other nodes.</p> <p>Another highly visible set of usecases would include "File/directory-per-process" kind of benchmarks like say mdtest.</p> LU-10938

Metadata writeback cache support

New Feature Major Open Unresolved Qian Yingjin Oleg Drokin Sun, 22 Apr 2018 02:27:39 +0000 Tue, 6 Dec 2022 00:25:07 +0000 0 21 <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32128" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32128</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Basic code to assimilate pagecache pages into clio<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: f3cdb29f8c02a88455bc1c9156fb70963e050cca</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32126" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32126</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Send and receive parent-locked lock handle<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 607a7170545314e5d381407c2e6c4990c557b290</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32124" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32124</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Basic symlink support - client and server<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8d0e49f970d8ada9af1a048192d79a705eec6898</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32116" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32116</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Intent mkdir implementation on client and server<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 30092387795a6d34bc8a30ec438f0022a05fed20</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32127" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32127</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Async file create: do intent open to create layout on the server<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: b482c5ace6c5201f6c8786b3625bf9bf2cec6ed2</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32121" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32121</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Add 40 RPCs at once limit<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 127a844d4cbd42d237976ec344215d387ba76314</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32119" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32119</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Server side of client-parent locked mkdir<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5b268fc0689fefa3b597779d19ed0b33817d4c8d</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32118" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32118</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Framework for WBC lock cancellation<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 6061281933204aa0f2f6ebb3d070880cbb2eb6f8</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32117" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32117</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Introduce LLIF_VIRTUAL flag for wbc-cached metadata<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 647f1567950e8f299f17f7cb9802ad5b75f971d9</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32120" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32120</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Client side of client-parent locked mkdir<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 055c497a3d1da9e5fd533a684be0e60732e052b0</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32125" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32125</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: chr/blockdev - pass in rdev to server for proper creation<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5a6eeee95d5c1cd5dca7f87b3ce0263d3fa7f668</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32123" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32123</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Added support for file creation<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 6750899f8b6b1a7b4d2b42984c5634d24701d83f</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32122" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32122</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: only request update bit on lookup speculatively with trybits<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8a59d672b252467dac9d4c76dbe22312ba4a8bf7</p> <p>Oleg,</p> <p>This is a bit of a silly question, but I don't know the answer in Gerrit.  How can I get the full patch series in to my local git/how can I tell what the order is so I can find the last one to check out?</p> <p>Thanks <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/wink.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>the tip of the tree is this patch: <a href="https://review.whamcloud.com/#/c/32128/3" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/32128/3</a> - you know this by looking in the right top part of that page titled "related changes" and seeing it's at the very top of the list.</p> <p>So if you checkout this patch (download -&gt; checkout) - you'll get this patch and everything else it's based upon</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32241" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32241</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> ptlrpc: Add WBC connect flag<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 7a30319c516f54f818e678483c08d8a30076aa15</p> <p>Thanks!</p> <p>I'll probably figure this out, but is there any particular trick to turning this on?  Will it happen automatically if I make a new directory and start creating stuff in it?</p> <p>yes, current code asumes you start from an mkdir, the resulting directory will have wbc fully enabled (there's a bit of a bug that if you delete the directory then create another with the same name - wbc would not turn on, I need to track it down some day).</p> <p>Perfect.  Just observed that myself.  Thank you, nice and easy to test. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/32241/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32241/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> ptlrpc: Add WBC connect flag<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: f024aabf8bbf797e101eec5ff054f7a6e3d253ed</p> <p>One thought I had was whether it is possible to cache/merge setattr-like requests like <tt>utimensat()</tt>, <tt>fchmod()</tt>, <tt>fchown()</tt>, <tt>fchgrp()</tt>, etc. on the client when creating/writing DoM regular files on the client? The client should already have a write lock on the file for DoM write, and the MDS shouldn't care whether the time, mode, permission, and maybe UID/GID changes before the file is written to the MDS because these attributes should all be sent with the write request.</p> <p>I should perhaps know the answer to this question, but which Lustre attributes are exclusively on the MDS?  Permissions are, xattrs are, and size is not (even in DOM, it's pulled from the data object - it's just that object is on the MDS) but what about the various time properties?</p> <p>The issue that will come up, I think, is which bits the client has a write lock on.</p> <p>The timestamps belong to the target that has the most recent ctime, but in any case since this optimization would be useful mostly for DoM files, the MDS would own all of the attributes including the size, as long as the later components were not initialized. Even if the later components were initialized, setting the timestamps, UID, GID, permission, etc. could be cached on the client as long as it has the DoM write lock on the MDT inode, since no other clients could modify/access these attributes.</p> <p>Status of the prototype, per <a href="https://hps.vi4io.org/_media/events/2019/hpc-iodc-lustre_next_20_years-dilger.pdf" class="external-link" target="_blank" rel="nofollow noopener">ISC</a></p> <ul> <li>No cache/quota/space limits, no background flushing, no batching, ...</li> <li>Early tests show 10-20x single-client speedup tests (untar, make, …)</li> </ul> <p>After reading the code, I found that current WBC still need to solve the following problem of reopen the file:</p> <ol> <li>When WBC is valid, open the file;</li> <li>the EX layout was revoked, the file is flushed</li> <li>at this time, we still need to reopen the file on MDS.</li> </ol> <p>I will try to rebase all WBC patches against master branch.<br/> Should I merge the series of WBC patches into a big one patch?</p> <p>I would strongly prefer <b>not</b> merging them.  Having them broken out like this is really nice.</p> <p>Yingjin Qian (qian@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/36851" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36851</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: Basic test scripts for WBC<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 97f122f36e995e251defe8f53a511d614e747ad2</p> <p>Yingjin Qian (qian@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/37104" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37104</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10938" title="Metadata writeback cache support" class="issue-link" data-issue-key="LU-10938">LU-10938</a> wbc: add symlink support<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 96f7534209e0c7c81b891ea0cadcdb9e9e746237</p> Related LU-13008 LU-13009 LU-13010 LU-13015 LU-13007 LU-13016 LU-13011 LU-13012 LU-16361 LU-14139 LU-13015 LU-11018 LU-14138 LU-13007 LU-13008 LU-13009 LU-13010 LU-13011 LU-13012 LU-13015 LU-13016 LU-13021 LU-13022 LU-13044 LU-13045 LU-13046 LU-13047 LU-13105 LU-13106 LU-13521 LU-13563 LU-13564 LU-13577 LU-14123 LU-14141 LU-14757 LU-14820 LU-14886 LU-15413 LU-15426 LU-15441 LU-15499 LU-15621 LU-15669 LU-15799 LU-15805 LU-16071 LU-16107 Development Epic/Theme Performance Rank 1|hzzw67: Rank (Obsolete) 9223372036854775807