https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5663 Lustre <p>Recently I ran mds-survey performance tests on master and b2_5 and found significantly performance regressions.</p> <p>The performance data are as follows:</p> <p>b2_5:</p> <div class='table-wrap'> <table class='confluenceTable'><tbody> <tr> <td class='confluenceTd'>test name </td> <td class='confluenceTd'>create</td> <td class='confluenceTd'>lookup</td> <td class='confluenceTd'>getattr</td> <td class='confluenceTd'>setxattr</td> <td class='confluenceTd'>destroy</td> </tr> <tr> <td class='confluenceTd'>p3700_sc0_32t_32dir</td> <td class='confluenceTd'>157343</td> <td class='confluenceTd'>1298292</td> <td class='confluenceTd'>876289</td> <td class='confluenceTd'>93022</td> <td class='confluenceTd'>143316</td> </tr> <tr> <td class='confluenceTd'>p3700_sc0_64t_32dir</td> <td class='confluenceTd'>170724</td> <td class='confluenceTd'>1378018</td> <td class='confluenceTd'>914515</td> <td class='confluenceTd'>99558</td> <td class='confluenceTd'>138258</td> </tr> <tr> <td class='confluenceTd'>p3700_sc1_32t_32dir</td> <td class='confluenceTd'>49900</td> <td class='confluenceTd'>1279590</td> <td class='confluenceTd'>881454</td> <td class='confluenceTd'>98467</td> <td class='confluenceTd'>33632</td> </tr> <tr> <td class='confluenceTd'>p3700_sc1_64t_32dir</td> <td class='confluenceTd'>40075</td> <td class='confluenceTd'>1274074</td> <td class='confluenceTd'>901117</td> <td class='confluenceTd'>100437</td> <td class='confluenceTd'>35839</td> </tr> </tbody></table> </div> <p>MASTER(pre-2.7)</p> <div class='table-wrap'> <table class='confluenceTable'><tbody> <tr> <td class='confluenceTd'>p3700_sc0_32t_32dir</td> <td class='confluenceTd'>95035</td> <td class='confluenceTd'>1124743</td> <td class='confluenceTd'>124594</td> <td class='confluenceTd'>76011</td> <td class='confluenceTd'>53653</td> </tr> <tr> <td class='confluenceTd'>p3700_sc0_64t_32dir</td> <td class='confluenceTd'>40043</td> <td class='confluenceTd'>1069693</td> <td class='confluenceTd'>49133</td> <td class='confluenceTd'>56457</td> <td class='confluenceTd'>51161</td> </tr> <tr> <td class='confluenceTd'>p3700_sc1_32t_32dir</td> <td class='confluenceTd'>29693</td> <td class='confluenceTd'>1106520</td> <td class='confluenceTd'>120479</td> <td class='confluenceTd'>60920</td> <td class='confluenceTd'>37890</td> </tr> <tr> <td class='confluenceTd'>p3700_sc1_64t_32dir</td> <td class='confluenceTd'>26208</td> <td class='confluenceTd'>1165051</td> <td class='confluenceTd'>383138</td> <td class='confluenceTd'>59853</td> <td class='confluenceTd'>38974</td> </tr> </tbody></table> </div> <p>PS: p3700 is the test name;<br/> sc0 means mds-survey stripe count 0 test<br/> 32t means 32 threads<br/> 32dir means 32 directories.</p> <p> Therefore p3700_sc1_32t_32dir refers to the test {create, lookup,getattr,setxattr,destroy} files with 1 stripe, and there are 32 threads doing the work against 32 different directories.</p> LU-5663

mds-survey performance regress on master

Bug Blocker Closed Cannot Reproduce WC Triage Jinshan Xiong Thu, 25 Sep 2014 01:43:15 +0000 Thu, 19 Feb 2015 23:16:09 +0000 Thu, 19 Feb 2015 23:16:09 +0000 Lustre 2.7.0 0 6 <p>This issue may be related <a href="https://jira.whamcloud.com/browse/LU-5621" title="Performance regression in 2.6 branch on file operations to shared directory " class="issue-link" data-issue-key="LU-5621"><del>LU-5621</del></a></p> <p>This is not a fix, but just remove those redundant stuff in the create object patch. Jinshan could you please try whether this patch can make master get back to 2.5. Thanks!</p> <p>You could try reverting <a href="http://review.whamcloud.com/10376" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10376</a> which is reducing the max transaction size by 1/2. </p> <p>Sorry I made a terrible mistake here. I reran the test but I didn't see any performance regression. I must have used different parameters when I was running the test against b2_5.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> Create Lookup Md_getattr Setxattr Destroy b2_5: p3700_sc0_32t_32dir 141313.71 1265631.61 887451.61 94046.43 131052.37 p3700_sc0_64t_32dir 170025.57 1337869.1 892227.34 103872.16 138735.83 p3700_sc1_32t_32dir 48335.6 1253868.18 876506.64 97518.53 32697.35 p3700_sc1_64t_32dir 39041.31 1257847.46 735008.57 101502.42 33928.42 Master: p3700_sc0_32t_32dir 138848.11 1263207.89 865148.7 88467.54 129140.86 p3700_sc0_64t_32dir 149196.48 1335493.91 875077.15 95105.77 129005.38 p3700_sc1_32t_32dir 48971.7 1237754.48 839588.74 97515.56 35833.2 p3700_sc1_64t_32dir 39285.28 1297680.39 839741.44 94257.2 32877.89 </pre> </div></div> <p>Here is the latest result. BTW, either reverting the patch or applying Di's patch didn't boost the performance.</p> <p>Is it possible you formatted the device differently when you ran the master test for 2.7? Maybe specifying a smaller device size or similar? If the MDT is formatted with a smaller size it could change the filesystem parameters (e.g. inode ratio, journal size, etc).</p> <p>Another possibility is that there was something else in the filesystem that caused it to run more slowly (e.g. files from some previous testing? Did you also format the OSTs identically for all of the tests?</p> <p>I reformatted OST every time for each test. Mostly likely I ran the pre-2.7 test with a smaller journal size.</p> Related LU-5621 Development Rank 1|hzwx3z: Rank (Obsolete) 15874 Severity