https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11739 Lustre <p>There is no need to inherit the default directory layout from the root directory when subdirectories are created therein. This consumes xattr space on the subdirectories, and makes it more complex to change the filesystem default layout in the future.</p> <p>Instead, it should be possible to check if the parent directory is the root directory and not copy the default layout xattr to the new subdirectory.</p> LU-11739

Don't inherit default layout from root for new subdirectories

Improvement Critical Resolved Fixed Jian Yu Andreas Dilger Fri, 7 Dec 2018 07:56:57 +0000 Tue, 20 Oct 2020 10:43:25 +0000 Fri, 4 Oct 2019 12:42:15 +0000 Lustre 2.12.0 Lustre 2.10.6 Lustre 2.13.0 Lustre 2.10.7 Lustre 2.12.4 0 9 <p>The change needs to be made in lod_ah_init() in lustre/lod/lod_object.c. I'm creating the patch.</p> <p>Jian Yu (yujian@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33956" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33956</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: don't inherit default layout from root directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a55f947c5a70ba6eb9fab077683dafbf1049a788</p> <p>In all cases, new subdirectories under root should <b>not</b> explicitly inherit the file layout from the root (we cannot specify the default file layout at the same time the directory is created, so your 1b) case above cannot happen, unless the default file layout is explicitly specified after the directory is created). </p> <p>This avoids the case where the filesystem-wide default layout is set on the root directory of a new filesystem, and then every subdirectory created in the filesystem explicitly inherits this filesystem-wide layout. Then, if the filesystem-wide "default" layout changes, none of the new files created use this new default because every parent directory already has an old explicitly inherited layout that is being used instead.</p> <p>We need the explicit inheritance behaviour for subdirectories because we can't walk the whole directory tree on every file create to find if some parent-of-parent has a default file layout that we should be inheriting. However, we should consider whether an explicit layout should be inherited from a parent directory that is exactly the same as the filesystem default layout? On the plus side, always inheriting the parent layout avoids some overhead to compare an explicit parent directory layout against the filesystem-wide default layout. On the minus side, this continues the issue that currently exists with explicitly-inherited default layouts. If we <b>don't</b> explicitly inherit a layout from the parent that matches the filesystem-wide default, it would slowly improve existing filesystems by preventing the spread of explicit-but-fs-default layouts.</p> <p>In such cases, it might make sense to add a tunable to control the "inherit filesystem-wide default file layout from parent" behaviour (e.g. <tt>lod.*.stripe_inherit_default</tt> or similar, but default the tunable to be off (which is IMHO the better long-term solution), and allow it to be enabled if this is the wrong choice for some reason.</p> <p>Hi Andreas,</p> <p>Before adding the implementation of lod.*.stripe_inherit_default, I did some tests to verify patch <a href="https://review.whamcloud.com/33956" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33956</a>. Here are the test results:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># getfattr -n trusted.lov /mnt/lustre /mnt/lustre: trusted.lov: No such attribute # mkdir /mnt/lustre/olddir # getfattr -n trusted.lov /mnt/lustre/olddir /mnt/lustre/olddir: trusted.lov: No such attribute # lfs setstripe -S 8M /mnt/lustre # getfattr -n trusted.lov /mnt/lustre getfattr: Removing leading '/' from absolute path names # file: mnt/lustre trusted.lov=0s0AvRCwEAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAA//8= # mkdir /mnt/lustre/newdir # getfattr -n trusted.lov /mnt/lustre/newdir /mnt/lustre/newdir: trusted.lov: No such attribute &lt;------ newdir didn't inherit root directory default layout # lfs setstripe -p pool /mnt/lustre # getfattr -n trusted.lov /mnt/lustre getfattr: Removing leading '/' from absolute path names # file: mnt/lustre trusted.lov=0s0AvTCwEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA//9wb29sAAAAAAAAAAAAAAAA # mkdir /mnt/lustre/newdir_pool # getfattr -n trusted.lov /mnt/lustre/newdir_pool /mnt/lustre/newdir_pool: trusted.lov: No such attribute &lt;------ newdir_pool didn't inherit root directory default layout # lfs setstripe -S 4M -p pool /mnt/lustre/newdir_pool # getfattr -n trusted.lov /mnt/lustre/newdir_pool getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/newdir_pool trusted.lov=0s0AvTCwEAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAA//9wb29sAAAAAAAAAAAAAAAA # mkdir /mnt/lustre/newdir_pool/subdir # getfattr -n trusted.lov /mnt/lustre/newdir_pool/subdir getfattr: Removing leading '/' from absolute path names # file: mnt/lustre/newdir_pool/subdir trusted.lov=0s0AvTCwEAAAAEAAAAAAAAAAEEAAACAAAAAABAAAAA//9wb29sAAAAAAAAAAAAAAAA &lt;------ subdir inherited non-root parent directory default layout </pre> </div></div> <p>Could you please take a look at the above test results to see if they are as expected? Thank you.</p> <p>Yes, I think this looks correct. </p> <p>There are references here to some sort of file system wide default layout.  I wasn't aware that was a possibility - Is there (or could someone give <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> ) a quick explanation of that and how to interact with it?  I just thought you had directory level defaults (optionally set by users/admins) and built-in system defaults (defined in the code).</p> <p>The old-style "filesystem wide" plain defaults (used in case everything else fails) are stored in the <tt>lov_descriptor</tt> for the filesystem from the <tt>mkfs.lustre --param lov.stripesize</tt> and similar options. They can also be set via <tt>lctl conf_param</tt> on the MGS.</p> <p>However, for quite a few releases, the more common interface for setting the filesystem-wide default is to use "<tt>lfs setstripe</tt>" as root to set a default layout on the root directory of the filesystem. This allows complex default layouts (e.g. pools, PFL, FLR, etc) to be set for the filesystem.</p> <p>This is mentioned in the man page for <tt>lfs-setstripe.1</tt>:</p> <blockquote> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>DESCRIPTION The lfs setstripe command is used to create a new file in a Lustre filesystem with the specified layout, or to specify the default layout for new files created in directory, or anywhere in the filesystem if directory is the filesystem root and no other layout takes precedence. Files with composite layouts allow different STRIPE_OPTIONS to be specified for non-overlapping extents of the file. Files will inherit options not explicitly specified on the command line either from the default layout on the parent directory, or from the filesystem-wide default. The default layout set on a directory will be copied to any new subdirectories created within that directory at the time they are created. </pre> </div></div></blockquote> <p>However, I note that this text needs to be updated to reflect that the filesystem default layout will not be explicitly copied to subdirectories under the root directory and their children to ensure that the filesystem default layout continues to be applied to those directories if the default is changed.</p> <p>Thanks, Andreas - Nice to have the background here.</p> <p>I think system-wide default layout needs a new way of definition. For example, we can use one bit in the layout to specify that a directory should apply system default layout(like a layout pointer).</p> <p>This patch makes root directory different from the other directories, which seems confusing from my point of view.</p> <p>Actually, it isn't really different than other directories in that they <b>do</b> inherit the layout from the parent, they just don't make a copy. That avoids copying the current default layout all the way from the root directory down to all the subdirectories and then if the default changes there are a thousand copies of the old default around the filesystem. </p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33956/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33956/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: don't inherit default layout from root directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 0a988cae95f99fee1a9c0d489ce00d0954d2a68e</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/34135" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34135</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: don't inherit default layout from root directory<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 2c55161aab2c6ed7cd0cf25b6d6b27a27bc9b932</p> <p>Jian Yu (yujian@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/34139" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34139</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: don't inherit default layout from root directory<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: 79cbd455b9d33d5942328046fbf62b0e7f83b73d</p> <p>Landed for 2.13</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/34139/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34139/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: don't inherit default layout from root directory<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: ff53b1a267feee05094414e6eb4b4cd75033e1e6</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/34135/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34135/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: don't inherit default layout from root directory<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 6ceb90a2461e2724121a6c26434e11bcb31c9f43</p> <p>This patch looks wrong to me: in lod_ah_init(), if parent directory is ROOT, it skips reading default layout, which means subdirs under ROOT don't follow default layout, while in my understanding it need to read default layout anyway, and then use it to decide child layout, but it doesn't inherit this default layout (by clearing it after use).</p> <p>Lai, could you please put your comments in the form of a test case, to verify if this is working or broken. If it is broken, then we can use your patch to verify it is working when it is fixed.</p> <p>Okay, I'll work on it after vacation.</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/35204" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35204</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: subdir under ROOT should honor default layout<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 3a615a0fbd184015e2aea4e5a02015f054a4698b</p> <p>There is a patch that has not been landed yet. </p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/35204/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35204/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: subdir under ROOT should honor default layout<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 693fb63ac777eab426f1b618316a5649534759ad</p> <p>Landed for 2.13</p> <p>Minh Diep (mdiep@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36370" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36370</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: subdir under ROOT should honor default layout<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: ba7427ac4dfbee42e34ba3b1b6a9a7b33e8c89dc</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36370/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36370/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11739" title="Don&#39;t inherit default layout from root for new subdirectories" class="issue-link" data-issue-key="LU-11739"><del>LU-11739</del></a> lod: subdir under ROOT should honor default layout<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: a53869aa31ceee700215520e5267b155a0f59d46</p> Duplicate LU-12433 Related LU-11875 LU-11995 LU-9982 LU-12643 LU-11656 LU-11920 Development Rank 1|i007in: Rank (Obsolete) 9223372036854775807