https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-7915 Lustre <p>During discussion with LBNL on their Hadoop Spark project, they said that there was considerable overhead when running on Lustre because of repeated open+close of the same files causing extra RPC traffic to the MDS.</p> <p>Lustre has the ability to cache opens on the client with a DLM openlock, but this isn't done for regular opens by default because it has extra overhead compared to uncached opens, but only for NFS opens because the knfsd repeatedly opens the same file.</p> <p>It would be worthwhile to firstly implement a tunable to enable opencache on a per-client basis (<a href="https://jira.whamcloud.com/browse/LU-5426" title="Add more controls for open file cache" class="issue-link" data-issue-key="LU-5426"><del>LU-5426</del></a>) and then measure the performance impact of this tunable for normal usage and for Spark.</p> LU-7915

investigate heuristics for SPARK client getting MDS openlock

Improvement Minor Open Unresolved Emoly Liu Andreas Dilger Thu, 24 Mar 2016 05:15:01 +0000 Thu, 28 May 2020 12:13:54 +0000 0 8 <p>Hi <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=adilger" class="user-hover" rel="adilger">adilger</a><br/> Could you please ask more details on their Spark project? I run benchmark test on Spark and I didn't notice this issue. I notice an extensive use of getxattr() solved by the new (2.5) lustre local xattr cache. Thank you</p> <p>Hi Emoly,</p> <p>Could you please have a look at measuring the performance as indicated by Andreas' final comment:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>It would be worthwhile to firstly implement a tunable to enable opencache on a per-client basis (LU-5426) and then measure the performance impact of this tunable for normal usage and for Spark. </pre> </div></div> <p>Thanks.<br/> Joe</p> <p>Can anybody show me any more details about this Spark performance testing? e.g. Spark version, benchmark tool, lustre version and how large scale. Thanks.</p> <p>The following pages describe their testing:</p> <p><a href="http://crd.lbl.gov/departments/computer-science/CLaSS/staff/costin-iancu/intel-parallel-computing-center-big-data-support-on-hpc-systems/" class="external-link" target="_blank" rel="nofollow noopener">http://crd.lbl.gov/departments/computer-science/CLaSS/staff/costin-iancu/intel-parallel-computing-center-big-data-support-on-hpc-systems/</a><br/> <a href="http://crd.lbl.gov/assets/pubs_presos/spark-on-cray.pdf" class="external-link" target="_blank" rel="nofollow noopener">http://crd.lbl.gov/assets/pubs_presos/spark-on-cray.pdf</a></p> <p>Emoly Liu (emoly.liu@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/19664" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/19664</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-7915" title="investigate heuristics for SPARK client getting MDS openlock " class="issue-link" data-issue-key="LU-7915">LU-7915</a> mdc: add a tunable to enable opencache<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 770f30f801417bcc24b39f0f58fe0abe35bc2bcd</p> Related LU-10948 LU-10948 LU-5426 Development Rank 1|hzy5kf: Rank (Obsolete) 9223372036854775807