https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-7251 Lustre <p>OSP can create a lot of commit callbacks to track changes to OST.</p> <p>also, due to current tracking mechanism, we allocate 732 in 3 chunks for every thandle and release up on commit. at 10K transactions/sec and 5s commit interval it gives ~35MB consumed. we can get rid of this.</p> LU-7251

reduce commit callbacks in OSP

Improvement Minor Resolved Fixed Alex Zhuravlev Alex Zhuravlev Sun, 4 Oct 2015 16:22:59 +0000 Wed, 13 Feb 2019 04:38:59 +0000 Wed, 1 Nov 2017 08:29:51 +0000 Lustre 2.11.0 Lustre 2.10.4 0 4 <p>Alex Zhuravlev (alexey.zhuravlev@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/17270" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/17270</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7251" title="reduce commit callbacks in OSP" class="issue-link" data-issue-key="LU-7251"><del>LU-7251</del></a> osp: do not assign commit callback to every thandle<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 34c397ae9abdfffd4febb07578b0adb056aef556</p> <p>we've tested patches and it's useful and improved performance a lot of bached metadata operations.<br/> Here is quick test resutls with/without patches. It's kind of metadata stress testing. mdtest for 2.56M files creation with 5 iterations. (mdtest -d /scratch/dir0 -u -v -i 5 -n 10000 from 32 clients, 256 processes)<br/> without patches, Directory creation and file removal are impacted by async operations for previous metadata tests (e.g a lo tof callback transation behind)</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>SUMMARY: (of 5 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- Directory creation: 42984.628 36953.381 39881.281 2292.603 Directory stat : 329218.490 322008.772 326116.622 2642.107 Directory removal : 92163.630 79497.765 85630.907 4297.749 File creation : 117720.781 83613.736 99194.213 12354.962 File stat : 328629.182 314377.011 318491.614 5447.589 File read : 256663.372 243983.658 249932.488 4027.999 File removal : 74525.772 54910.856 65667.784 7710.949 Tree creation : 256.674 70.697 199.439 67.303 Tree removal : 8.069 7.260 7.550 0.285 V-1: Entering timestamp... </pre> </div></div> <p>With patches, Dir creation and File remove imporved a lot and it's stable results.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>SUMMARY: (of 5 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- Directory creation: 48695.989 41188.882 44081.677 2834.035 Directory stat : 310020.068 296526.272 303988.450 4788.302 Directory removal : 93812.856 80760.823 88067.584 4621.696 File creation : 123718.206 91192.211 101872.614 11548.931 File stat : 307580.997 295884.726 302662.073 4837.231 File read : 273169.928 221255.575 258596.002 20265.696 File removal : 105689.008 88411.461 98718.108 6047.066 Tree creation : 252.958 36.398 180.527 78.435 Tree removal : 7.638 0.704 4.741 3.283 V-1: Entering timestamp... </pre> </div></div> <p>I've tested latest patch (patchset 29) against latest master and no regressions at all.</p> <p>master without patch </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>mpirun mdtest -n 10000 -u -v -d /scratch0/mdtest.out -F -i 3 -p 10 -w 0 32 client, 128 processes SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 114523.666 108810.588 111770.237 2336.908 File stat : 303861.727 288229.127 294945.438 6568.839 File read : 162822.391 153932.256 157040.416 4092.317 File removal : 127130.002 116283.786 119919.433 5098.703 Tree creation : 420.030 180.980 317.848 100.626 Tree removal : 12.986 11.249 12.007 0.726 V-1: Entering timestamp... mpirun mdtest -n 10000 -u -v -d /scratch0/mdtest.out -F -i 3 -p 10 -w 1048576 32 client, 128 processes SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 41194.158 35728.404 37847.067 2394.493 File stat : 310838.872 307811.966 309721.094 1356.519 File read : 165343.510 164597.509 164868.278 337.128 File removal : 117608.944 115636.150 116642.341 805.876 Tree creation : 462.102 135.079 323.501 138.076 Tree removal : 12.555 11.170 11.884 0.566 V-1: Entering timestamp... </pre> </div></div> <p>with patch</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>mpirun mdtest -n 10000 -u -v -d /scratch0/mdtest.out -F -i 3 -p 10 -w 0 32 client, 128 processes SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 128146.138 112097.493 119258.116 6664.717 File stat : 306953.378 299898.991 304338.726 3155.872 File read : 165972.464 160745.027 164083.325 2367.357 File removal : 142539.301 123405.006 135905.318 8844.609 Tree creation : 424.966 98.191 287.812 138.468 Tree removal : 13.934 11.025 12.419 1.190 V-1: Entering timestamp... -- finished at 09/12/2017 08:30:01 -- mpirun mdtest -n 10000 -u -v -d /scratch0/mdtest.out -F -i 3 -p 10 -w 1048576 32 client, 128 processes SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 41306.861 40326.870 40692.381 437.100 File stat : 316061.726 311079.602 314214.401 2228.390 File read : 166757.778 164512.030 165589.820 919.037 File removal : 140835.710 113936.502 128198.607 11041.507 Tree creation : 479.815 190.251 372.237 129.394 Tree removal : 12.696 10.593 11.918 0.942 V-1: Entering timestamp... </pre> </div></div> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/17270/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/17270/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7251" title="reduce commit callbacks in OSP" class="issue-link" data-issue-key="LU-7251"><del>LU-7251</del></a> osp: do not assign commit callback to every thandle<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 0ba690a526be74c4cdffe7a7dd3031b4bd2b37d8</p> <p>Landed for 2.11</p> <p>Minh Diep (minh.diep@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/29879" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29879</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7251" title="reduce commit callbacks in OSP" class="issue-link" data-issue-key="LU-7251"><del>LU-7251</del></a> osp: do not assign commit callback to every thandle<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: f7a99006cab0235872c6af165a016142c669d7f6</p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/29879/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29879/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7251" title="reduce commit callbacks in OSP" class="issue-link" data-issue-key="LU-7251"><del>LU-7251</del></a> osp: do not assign commit callback to every thandle<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: 236f73509cdcc83cdb56cdea376ff4a4e7f378c7</p> Blocker LU-7895 Duplicate LU-10170 Related LU-10066 LU-10230 Development Rank 1|hzxpin: Rank (Obsolete) 9223372036854775807