https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11376 Lustre <p>Add support for a new extended attribute or special file/dir layout to represent in the Lustre namespace an external DAOS container identified by a UUID. The pro’s and con’s of both options will be presented at the LAD’18 and summarized in this ticket.</p> <p>The purpose of this ticket is to track the effort to implement this feature.</p> LU-11376

Special file/dir to represent DAOS Containers

New Feature Minor Resolved Fixed Bruno Faccini Johann Lombardi patch Fri, 14 Sep 2018 11:05:21 +0000 Wed, 10 Jan 2024 08:17:07 +0000 Wed, 8 May 2019 11:52:10 +0000 Lustre 2.13.0 0 14 <p>Special layout please, able to specify an arbitrary external reference. See <a href="https://jira.whamcloud.com/browse/LU-10606" title="HSM info as part of LOV layout xattr" class="issue-link" data-issue-key="LU-10606">LU-10606</a>.</p> <p>Faccini Bruno (bruno.faccini@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/33755" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33755</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11376" title="Special file/dir to represent DAOS Containers" class="issue-link" data-issue-key="LU-11376"><del>LU-11376</del></a> lov: new foreign LOV format<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 7b0a57a27e291cef71b7af0be6150ae0a3224189</p> <p>Gerrit change #33755 is a patch to allow for a file's flexible LOV format that will permit to specify an arbitrary external reference for a file. The idea behind this is to provide Lustre namespace support and LOV prefetch/caching under layout-lock protection, for user-land/external usage.</p> <p> </p> <p> </p> <p>Faccini Bruno (bruno.faccini@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/34087" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34087</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11376" title="Special file/dir to represent DAOS Containers" class="issue-link" data-issue-key="LU-11376"><del>LU-11376</del></a> lmv: new foreign LMV format<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 63ee0812223af72c39df776c7930dbb9cf791595</p> <p>Gerrit change #34087 is a patch to allow for a dir's flexible LMV format that will permit to specify an arbitrary external reference for a dir. The idea behind this is to provide Lustre namespace support and LMV prefetch/caching under lock protection, for user-land/external usage.</p> <p> </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-comment">/* foreign LMV EA */</span> struct lmv_foreign_md { __u32 lfm_magic; <span class="code-comment">/* magic number = LOV_MAGIC_FOREIGN */</span> __u32 lfm_length; <span class="code-comment">/* length of lfm_value */</span> <span class="code-object">char</span> lfm_value[]; }; </pre> </div></div> <p>Can we add fields here for __u32 lfm_type and __u32 lfm_flags so that Lustre can in the future have some way of understanding the contents? If everything is hidden inside lfm_value, then Lustre would have no way of, for example:</p> <ul> <li>selecting a client-side handler/upcall for different foreign types</li> <li>selecting a server-side handler of any type</li> <li>invoking the HSM mechanisms to automatically populate/swap-in a file</li> </ul> <p>Thanks Nathan. Unless this is a requirement to land this patch, I would rather address those changes in follow-on patches.</p> <blockquote><p>I would rather address those changes in follow-on patches.</p></blockquote> <p>I get the desire to both keep it simple and minimize churn, but the (potential) price to pay is future code complexity for compat checking. Since this is an on-disk structure, we will bear that burden into the eternal future. It's very cheap to do this now (literally, just add the fields - don't even need to bother with swapping them today), very expensive to do it later. Are there concrete plans to do something with these new fields? No. But it seems like it could have value in letting Lustre make some decisions about where/when/how such external files are handled.<br/> @adilger do you want to weigh in?</p> <p>I think the addition of two fields into the data structure for forward compatibility is not a huge burden, and I agree with Nathan that this is easily done now and harder to do later. Using those fields for various purposes can be deferred to later patches when that functionality is needed. </p> <p>Nathan, Andreas, this sounds reasonable, I will add those fields now.</p> <p>In patch-set #10 of Gerrit Change #33755, new __u32 fields lmf_type and lmf_flags have been added to foreign LOV format.</p> <p>In patch-set #8 of Gerrit Change #34087, new __u32 fields lmf_type and lmf_flags have been added to foreign LMV format.</p> <p> </p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33755/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33755/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11376" title="Special file/dir to represent DAOS Containers" class="issue-link" data-issue-key="LU-11376"><del>LU-11376</del></a> lov: new foreign LOV format<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 6a20bdcc608bc2b933774b9f34ec25395e920a54</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/34087/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34087/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11376" title="Special file/dir to represent DAOS Containers" class="issue-link" data-issue-key="LU-11376"><del>LU-11376</del></a> lmv: new foreign LMV format<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: fdad38781cccb05c4cf3f1458c2d2d7c8b2b5bec</p> <p>Landed for 2.13</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/34851" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34851</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11376" title="Special file/dir to represent DAOS Containers" class="issue-link" data-issue-key="LU-11376"><del>LU-11376</del></a> tests: skip sanity test_27K until it works<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: b00742d9ca6b29dfd7d1de737d36dee3068ba995</p> Related LU-7207 LU-12104 LUDOC-465 LU-10606 LU-12682 Development Rank 1|i002ef: Rank (Obsolete) 9223372036854775807