https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11656 Lustre <p>Running "lfs getstripe" on a directory only shows the simple filesystem default layout and not the root default layout that will actually be inherited by new files.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># lfs setstripe -p ssd /mnt/testfs # lfs getstripe -d /mnt/testfs stripe_count: 1 stripe_size: 1048576 stripe_offset: -1 pool: ssd # mkdir /mnt/testfs/tmp/should_be_old # lfs getstripe -d /mnt/testfs/tmp/should_be_old stripe_count: 1 stripe_size: 1048576 stripe_offset: -1 </pre> </div></div> LU-11656

"lfs getstripe" on directory does not show default root layout

Bug Major Resolved Fixed Jian Yu Andreas Dilger Sun, 11 Nov 2018 17:03:05 +0000 Sat, 27 Jun 2020 08:37:07 +0000 Fri, 20 Dec 2019 03:52:06 +0000 Lustre 2.12.0 Lustre 2.10.8 Lustre 2.14.0 Lustre 2.12.4 0 6 <p>Here is the test result on Lustre 2.10.5_ddn6 and master branch:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># lfs getstripe -d /mnt/lustre stripe_count: 1 stripe_size: 1048576 pattern: 0 stripe_offset: -1 # mkdir /mnt/lustre/olddir # lfs getstripe -d /mnt/lustre/olddir stripe_count: 1 stripe_size: 1048576 pattern: 0 stripe_offset: -1 # lfs setstripe -p OLD /mnt/lustre # lfs getstripe -d /mnt/lustre stripe_count: 1 stripe_size: 1048576 pattern: raid0 stripe_offset: -1 pool: OLD # lfs getstripe -d /mnt/lustre/olddir stripe_count: 1 stripe_size: 1048576 pattern: 0 stripe_offset: -1 # mkdir /mnt/lustre/newdir # lfs getstripe -d /mnt/lustre/newdir stripe_count: 1 stripe_size: 1048576 pattern: raid0 stripe_offset: -1 pool: OLD </pre> </div></div> <p>The result is different from the description part of this ticket. Actually, running "lfs getstripe" on newly created directory after setting pool name on its parent directory can still show the pool name. Only the directory created before setting pool name on its parent directory has pool name printing issue. </p> <p>While running "lfs getstripe", the following codes produced the different outputs for "/mnt/lustre" and "/mnt/lustre/olddir":</p> <div class="code panel" style="border-width: 1px;"><div class="codeHeader panelHeader" style="border-bottom-width: 1px;"><b>lov_dump_plain_user_lmm()</b></div><div class="codeContent panelContent"> <pre class="code-java"> __u32 magic = *(__u32 *)&amp;param-&gt;fp_lmd-&gt;lmd_lmm; <span class="code-comment">//...... </span> <span class="code-keyword">if</span> (magic == LOV_USER_MAGIC_V1) { &lt;------ <span class="code-quote">"/mnt/lustre/olddir"</span> had LOV_USER_MAGIC_V1 lov_dump_user_lmm_v1v3(...); } <span class="code-keyword">else</span> { &lt;------ <span class="code-quote">"/mnt/lustre"</span> had LOV_USER_MAGIC_V3 <span class="code-object">char</span> pool_name[LOV_MAXPOOLNAME + 1]; struct lov_user_ost_data_v1 *objects; struct lov_user_md_v3 *lmmv3 = (void *)&amp;param-&gt;fp_lmd-&gt;lmd_lmm; snprintf(pool_name, sizeof(pool_name), <span class="code-quote">"%s"</span>, lmmv3-&gt;lmm_pool_name); objects = lmmv3-&gt;lmm_objects; lov_dump_user_lmm_v1v3(...); } </pre> </div></div> <p>After running "lfs setstripe -p OLD" on "/mnt/lustre", its magic was changed from LOV_USER_MAGIC_V1 to LOV_USER_MAGIC_V3 in lod_xattr_set(). However, the previously created "/mnt/lustre/olddir" still had magic of LOV_USER_MAGIC_V1 and was not changed.</p> <p>After <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a>, we shouldn't be explicitly inheriting the layout from the root directory anymore.</p> <p>However, I think the "lfs getstripe" on directories that do not have an actual LOV EA will only print the stripe_count, stripe_size, and stripe_index that is fetched from the /sys/fs/lustre/lov values.</p> <p>We need some way to fetch the actual default layout that will be used when new files will be created in that directory. We can't use the layout from the directory of the mountpoint, since that might be a subdirectory mount. One option would be to fetch the layout from the ROOT FID?</p> <p>Jian, I wanted to make sure the "explicitly inheriting" issue was clear, as it explains why your test showed different results than the original bug report.  You did everything in the top-level.  So, to reproduce and see that "lfs getstripe" does not actually report the filesystem default, try something like this:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># lfs getstripe -d /lfstmp stripe_count: 1 stripe_size: 1048576 stripe_offset: -1 pool: OLD # mkdir /lfstmp/toplevel # lfs getstripe -d /lfstmp/toplevel stripe_count: 1 stripe_size: 1048576 stripe_offset: -1 pool: OLD (that is the root level Explicit inheritance) # lfs setstripe -d /lfstmp/toplevel # lfs getstripe -d /lfstmp/toplevel stripe_count: 1 stripe_size: 1048576 stripe_offset: -1 # mkdir /lfstmp/toplevel/should_be_old # lfs getstripe -d /lfstmp/toplevel/should_be_old/ stripe_count: 1 stripe_size: 1048576 stripe_offset: -1 (neither the top level or new dirs display "OLD", but they should) </pre> </div></div> <p>Hope this helps!<br/> -Nathan</p> <p>Thank you, Nathan.</p> <p>However, "lfs setstripe -d /lfstmp/toplevel" will delete both the striping pattern and OST pool (fixed in <a href="https://jira.whamcloud.com/browse/LU-10629" title="Cannot clear a default OST pool from root" class="issue-link" data-issue-key="LU-10629"><del>LU-10629</del></a>) on the directory. So, the above test results are correct.</p> <p>Jian.</p> <p>I just wanted to make sure you were aware that this is still a problem, even with <a href="https://jira.whamcloud.com/browse/LU-10629" title="Cannot clear a default OST pool from root" class="issue-link" data-issue-key="LU-10629"><del>LU-10629</del></a> fixed or deleting the stripe info on a directory so that it should implicitly inherit the filesystem default.  I agree with the statement that Andreas made "<em>We need some way to fetch the actual default layout that will be used when new files will be created in that directory.</em>"</p> <p>Thanks,<br/> Nathan</p> <p>Hi Nathan,<br/> Let me look into this and work on it.</p> <p>According to Andreas' comments, I'm working on the patch to fetch the layout from the ROOT FID <span class="error">&#91;FID_SEQ_ROOT, FID_OID_ROOT, 0&#93;</span>.</p> <p>Jian, is the default layout for the directory going to be fetched from the ROOT FID inside the kernel or from the "<tt>lfs getstripe</tt>" command? Fetching it from the kernel simplifies some issues for userspace applications, but has the drawback that tools like "tar" will make a copy of the default <tt>trusted.lov</tt> xattr for every directory in the filesystem which doesn't have its own xattr. </p> <p>Doing it in "<tt>lfs getstripe</tt>" avoids this issue, but may have other issues (e.g. opening <tt>$MOUNT/.lustre/fid</tt> is only accessible to root, and is not available in subdirectory mounts.</p> <p>Hi Andreas,</p> <p>As per your suggestions, fetching default layout for a directory will be done in lfs_getstripe()&#45;&gt;...&#45;&gt;cb_getstripe(). If get_lmd_info() returns -ENODATA for a directory that does not have trusted.lov xattr, cb_getstripe() should get the lov_user_md data from ROOT FID, and then print the layout information in llapi_lov_dump_user_lmm().</p> <blockquote> <p>One solution would be to special-case the <span class="error">&#91;FID_SEQ_ROOT, FID_OID_ROOT, 0&#93;</span> lookup in the directory in ioctl(IOC_MDC_GETFILESTRIPE) to convert this ioctl onto the actual ROOT FID, since this is out of the critical code path, and is unlikely to cause any problems for existing files. It should treat -ENOENT as -ENODATA, in case it is running on an old client that does not have this feature, and it will just keep the same behaviour as today.</p> <p>That will probably involve taking the core of ll_lov_getstripe_ea_info() into a separate helper function that allows passing the ROOT FID, while the original ll_lov_getstripe_ea_info() gets the FID from inode.</p> <p>One upcoming issue with explicitly using the ROOT FID is that this will complicate things is the patch <a href="https://review.whamcloud.com/28972" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28972</a> "<a href="https://jira.whamcloud.com/browse/LU-9982" title="Clients striping from mapped FID in nodemap" class="issue-link" data-issue-key="LU-9982">LU-9982</a> lustre: Clients striping from mapped FID in nodemap", since that feature will allow clients to inherit layouts from a non-ROOT directory based on their nodemap. In that case, it might make sense to handle the ROOT FID getstripe to be mapped to the nodemap root layout (if set) so that the "lfs getstripe" output is consistent.</p></blockquote> <p>Jian Yu (yujian@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36609" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36609</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11656" title="&quot;lfs getstripe&quot; on directory does not show default root layout" class="issue-link" data-issue-key="LU-11656"><del>LU-11656</del></a> utils: fix "lfs getstripe" to fetch default layout for a directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 497c3bd1bd884f4f2c60870a096b19c1a8ceb9fd</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36609/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36609/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11656" title="&quot;lfs getstripe&quot; on directory does not show default root layout" class="issue-link" data-issue-key="LU-11656"><del>LU-11656</del></a> llite: fetch default layout for a directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3e8fa8a7396cd029cb0d7714a324343eed7f535e</p> <p>Landed for 2.14</p> <p>Minh Diep (mdiep@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/37072" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37072</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11656" title="&quot;lfs getstripe&quot; on directory does not show default root layout" class="issue-link" data-issue-key="LU-11656"><del>LU-11656</del></a> llite: fetch default layout for a directory<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: b4a8724c893ab66d4343dc68fbd00ebd1d561a32</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37072/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37072/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11656" title="&quot;lfs getstripe&quot; on directory does not show default root layout" class="issue-link" data-issue-key="LU-11656"><del>LU-11656</del></a> llite: fetch default layout for a directory<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: ca373442fc15a6dc77dcce36e91847c58ac45c03</p> Blocker LU-11918 Related LU-9982 LU-11739 LU-12643 LU-13687 Development Rank 1|i0067z: Rank (Obsolete) 9223372036854775807 Severity