https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9221 Lustre <p>Currently jobstats gets information by either dedicating an entire node to a single job (<a href="https://jira.whamcloud.com/browse/LU-7195" title="Allow for static string content for jobstats jobid_var" class="issue-link" data-issue-key="LU-7195"><del>LU-7195</del></a>) or reading from the userspace environment for each call. The first is very fast, but can be inaccurate, the second introduces a significant performance penalty on the client. This patch creates a cache of JobID names mapped to PIDs. It has several fallback mechanisms in order to present the most accurate info available.</p> LU-9221

Create pid-based hash to enhance Jobstats performance

Improvement Minor Resolved Fixed Ben Evans Ben Evans Thu, 16 Mar 2017 16:04:42 +0000 Thu, 25 Apr 2019 14:28:47 +0000 Thu, 21 Sep 2017 12:18:08 +0000 Lustre 2.9.0 Lustre 2.11.0 0 10 <p>Patch up for review is at <a href="https://review.whamcloud.com/#/c/25208/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/25208/</a>.</p> <p>@hdoreau, the OpenSFS LWG is looking to you to be the reviewer of record for this enhancement. Can you accept? We're getting close to code cut-off for v2.10 and we were hoping that you would have time for reviews. Thanks!</p> <p>Could still make 2.10 if reviews complete in time but otherwise will be in 2.10.1</p> <p>@cory: I'd be glad to help but it will not have much time to review Ben's patch before LUG...</p> <p>As seen in first revisions of the patch, introducing such a new cache comes with a risk of subtle race conditions and memory leaks. Therefore reviewing and testing it seriously cannot be done in a rush.</p> <p> <span class="nobr"><a href="https://jira.whamcloud.com/secure/attachment/26818/26818_JobID+Cache+Proposal.docx" title="JobID Cache Proposal.docx attached to LU-9221">JobID Cache Proposal.docx^{<img class="rendericon" src="https://jira.whamcloud.com/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> <br/> Updated doc</p> <p><a href="https://jira.hpdd.intel.com/browse/LUDOC-381" class="external-link" target="_blank" rel="nofollow noopener">https://jira.hpdd.intel.com/browse/LUDOC-381</a></p> <p>Another interesting tool which might be useful to implement this feature is the linux key management system (see man keyctl).</p> <p>One can create a key with some data attached and link it to one of the available keyrings (thread, process, session, default session, etc). The key will be stored on a temporal or permanent basis. The cache can be easily searched from the kernel using request_key().</p> <p>It's also possible to create a key type so that requested keys are created using a userspace upcall (see /sbin/request-key, /etc/request-key.conf).</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/25208/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/25208/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9221" title="Create pid-based hash to enhance Jobstats performance" class="issue-link" data-issue-key="LU-9221"><del>LU-9221</del></a> jobstats: Create a pid-based hash for jobid values<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 08479b74ec3599ee91e14f3f646389bb0aca4575</p> <p>Landed for 2.11</p> Related LUDOC-381 Development Epic/Theme Performance Rank 1|hzz7av: Rank (Obsolete) 9223372036854775807