https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5108 Lustre <p>To remember my recent work on performance tune</p> LU-5108

osc: Performance tune for LRU

Improvement Major Resolved Fixed Jinshan Xiong Jinshan Xiong Tue, 27 May 2014 21:33:31 +0000 Thu, 9 Feb 2017 17:18:06 +0000 Tue, 18 Aug 2015 13:55:25 +0000 Lustre 2.6.0 Lustre 2.8.0 0 8 <p>patch is at: <a href="http://review.whamcloud.com/10458" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10458</a></p> <p>patch <a href="http://review.whamcloud.com/10503" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10503</a> to add a per open file cache is spun off from the above patch.</p> <p>Several further things we can work on to reduce CPU usage on the client side.<br/> 1. cl_req - it maintains an inflight pages list which is not needed;<br/> 2. dirty grant - the interface is still page oriented, now that we can commit pages in batch, see osc_io_commit_async, we should work out new interfaces to reserve and release grant multiple pages based;<br/> 3. operations in cl_page - some operations can be provided in callback, for example, make_ready, prep, own and disown, etc, because they are VFS functionalities and LOV and OSC layer can't do anything for it. This way it can avoid looping page layers;<br/> 4. two many lists in oap - the side effect is that it has to loop the pages to move them between lists, also it consumes extra per-page memory. This can be avoided if we can reuse the same list entry for different purpose. For example, oap_</p> {pending, leu} <p>_item and ops_lru can share the same list because busy pages should not be in LRU list.</p> <p>Bobijam</p> <p>Could you please assist with this issue?</p> <p>Thanks</p> <p>Peter</p> <p>Set patch to blocker so that patch 10458 and 10503 can be landed in 2.6</p> <p>Jinshan, I think you are doing great work here! I am impressed with the results.</p> <p>However, Lustre is currently in code freeze, and Lustre 2.6.0 is already overdue. This is not the time to be landing new performance improvements. Especially for CLIO problems that, I assume, have been with us for years now. And ones that add new kernel threads.</p> <p>Furthermore, unless this was a <em>very</em> recent and <em>very</em> significant performance regression that <em>really</em> needs to block the 2.6.0 release, marking this ticket as a "blocker" is an abuse of process.</p> <p>So again, great work here. But I this should wait to land on master until after we have created the b2_6 branch.</p> <p>Chris</p> <p>We had already marked this as FixVersion 2.7 <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Peter</p> <p>Sure, but priority is still at the Blocker setting, which does not seem appropriate. And one of the other patches from this ticket landed only yesterday. In the absence of a comment (since the only comment we see is one requesting landing for 2.6), your intent was not particularly clear to those of us on the outside.</p> <p>ok. I have dropped the priority</p> <p>Your edit to the comment interleaved with my last comment. Yes you are right that a comment would have been more clear given the above comment. The intention is to track this work for 2.7. The patch that landed yesterday was quite minor and was considered low risk. Until code freeze is in effect there are other low risk changes still landing.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/10458/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10458/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5108" title="osc: Performance tune for LRU" class="issue-link" data-issue-key="LU-5108"><del>LU-5108</del></a> osc: Performance tune for LRU<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b117bc837c02e2d156bb114142a28a184aa9d633</p> <p>Landed for 2.8</p> Related LU-7951 LU-5260 LU-6643 LU-5291 Development Rank 1|hzwn5j: Rank (Obsolete) 14093