https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11623 Lustre <p>Currently when creating/opening a file, we do not let the client hash the name so next access (frequently stat?) goes to talk on the MDS. This is also seen in various tests like mdtest.</p> <p>We should consider returning more bits from the open call. This is in addition to <a href="https://jira.whamcloud.com/browse/LU-10948" title="client cache open lock after N opens" class="issue-link" data-issue-key="LU-10948">LU-10948</a> that handles similar situation for recurring non-creating opens.</p> LU-11623

Allow caching of open-created dentries

Improvement Major Reopened Unresolved Lai Siyao Oleg Drokin Tue, 6 Nov 2018 00:24:06 +0000 Mon, 22 Jan 2024 15:48:56 +0000 Lustre 2.10.0 Lustre 2.11.0 Lustre 2.14.0 Lustre 2.12.6 Lustre 2.16.0 1 11 <p>Oleg Drokin (green@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33584" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33584</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> llite: hash jsut created files if lock allows<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a72ed0f38312f3a036a51acdd4f0a9e93b1b3228</p> <p>Oleg Drokin (green@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33585" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33585</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> mdt: Opportunistically return UPDATE and PERM bits on open<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 1a59d230c86e5894d2d1148f786b9fa37ab904c5</p> <p><a href="https://review.whamcloud.com/#/c/32157" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/32157</a><br/> <a href="https://review.whamcloud.com/#/c/33584" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/33584</a><br/> <a href="https://review.whamcloud.com/#/c/33585" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/33585</a></p> <p>The above three patches(mainly two patches 32157 and 33585) can significant improve 'stat' operation at one of workload that is open-create, then stat on same client. <br/> This is one of common workload many people are doing on a single client. (e.g. create many files, then 'ls' command just for created files)<br/> Here is test results with/without those three patches. <br/> Test case is simple that create 240,000 x 32K files into a single directory, then 'ls -l'. I also ran mdtest in parallel for 'create' and 'stat' for those 1.2M files in addition to 'ls -l' test.</p> <p>Without patch(master branch commit 26a7abe)</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@c01 ~]# mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 10000 -d /cache1/out/ -F -C -v -w 32k [root@c01 ~]# time ls -l /cache1/out/#test-dir.0-0/mdtest_tree.0 &gt; /dev/null real 0m23.447s user 0m1.010s sys 0m11.916s [root@c01 ~]# mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d /cache1/out/ -F -C -T -v -w 32k SUMMARY rate: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 3838.205 3838.204 3838.204 0.000 File stat : 33459.289 33459.249 33459.271 0.011 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 3146.841 3146.841 3146.841 0.000 Tree removal : 0.000 0.000 0.000 0.000 </pre> </div></div> <p>With three patches agianst master (commit 26a7abe)</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@c01 ~]# mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 10000 -d /cache1/out/ -F -C -v -w 32k [root@c01 ~]# time ls -l /cache1/out/#test-dir.0-0/mdtest_tree.0 &gt; /dev/null real 0m5.780s user 0m0.635s sys 0m4.996s [root@c01 ~]# mpirun -np 24 --allow-run-as-root /work/tools/bin/mdtest -n 50000 -d /cache1/out/ -F -C -T -v -w 32k SUMMARY rate: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 3822.440 3822.439 3822.440 0.000 File stat : 350620.140 350615.980 350617.193 1.051 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 2076.727 2076.727 2076.727 0.000 Tree removal : 0.000 0.000 0.000 0.000 </pre> </div></div> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33584/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33584/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> llite: hash just created files if lock allows<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: fc42cbe0e2e5d1d87d0edca73986b831ac718301</p> <p>Oleg Drokin (green@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/35067" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35067</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> tests: Fix sanity 27E to ensure getattr RPC<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5f8705f8164e4cdaf22d96e5b37ae1bd362b7d3f</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/35067/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35067/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> tests: Fix sanity 27E to ensure getattr RPC<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 892c2251d854f480c272ab2999cdc6f4c87fecf0</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/38750" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38750</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> llite: hash just created files if lock allows<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 72ba58af89946e0e4b3743262bacc6af315483fa</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/38763" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38763</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> llite: hash just created files if lock allows<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 1922e2b202c9758f60dd1d72b2099302c0c0809c</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38763/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38763/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> llite: hash just created files if lock allows<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 7554658056cddae71fd90ef4a6dd639ddbe3f89b</p> <p>Do I understand correctly that the work tracked under this ticket landed to 2.13 and 2.12.6?</p> <p>Landed for 2.14 and 2.12.6</p> <p>I would open this ticket since patch <a href="https://review.whamcloud.com/#/c/33585" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/33585</a> was not merged yet. without this patch, what I showed the performance numbers can't be demonstrated. </p> <p><del>"Lai Siyao &lt;lai.siyao@whamcloud.com&gt;" uploaded a new patch:</del> <a href="https://review.whamcloud.com/47163" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47163</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> mdt: return UPDATE|PERM on open</del><br/> <del>Project: fs/lustre-release</del><br/> <del>Branch: master</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: 509656f89bd967395da859ea123875ca5db9931c</del></p> <p>latest patch set (patch set 29) of <a href="https://review.whamcloud.com/#/c/fs/lustre-release/+/33585/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/fs/lustre-release/+/33585/</a> patch still demonstrated and promised 9x (86,466 ops vs 789,729 ops) performance boost on open()-create()-stat() workload. And this is same as we are seeing same cached performance on stat()-stat() workload toady. Here is performance comparisons with/without patch.</p> <p>without patch(commit:8beeec77c3)</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -C -v -w 32k mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k SUMMARY rate: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 0.000 0.000 0.000 0.000 File stat : 86466.293 86466.272 86466.285 0.006 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 0.000 0.000 0.000 0.000 Tree removal : 0.000 0.000 0.000 0.000 mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 0.000 0.000 0.000 0.000 File stat : 779470.812 779428.353 779442.415 18.312 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 0.000 0.000 0.000 0.000 Tree removal : 0.000 0.000 0.000 0.000 </pre> </div></div> <p>With patch (<a href="https://review.whamcloud.com/#/c/fs/lustre-release/+/33585/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/fs/lustre-release/+/33585/</a>)</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -C -v -w 32k mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 0.000 0.000 0.000 0.000 File stat : 789729.392 789726.612 789727.879 0.702 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 0.000 0.000 0.000 0.000 Tree removal : 0.000 0.000 0.000 0.000 mpirun -np 16 mdtest -n 10000 -d /exafs/d0/d1/d2/ -F -T -v -w 32k Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 0.000 0.000 0.000 0.000 File stat : 790002.623 790000.712 790001.644 0.633 File read : 0.000 0.000 0.000 0.000 File removal : 0.000 0.000 0.000 0.000 Tree creation : 0.000 0.000 0.000 0.000 Tree removal : 0.000 0.000 0.000 0.000 </pre> </div></div> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/33585/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/33585/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11623" title="Allow caching of open-created dentries" class="issue-link" data-issue-key="LU-11623">LU-11623</a> mdt: return UPDATE|PERM on open<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 4b44204930cf4d312425dd95e51720fe5d3e801b</p> <p>Latest patch merged for 2.16. TBH it seems like it would have been better to track this latest work under a new linked Jira than to use this several years old ticket.</p> Duplicate Related LU-10948 LU-15725 Development Rank 1|i005rr: Rank (Obsolete) 9223372036854775807