https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11510 Lustre <p>The <tt>lfs_migrate</tt> tool does not currently preserve the PFL/FLR/DoM components of the layout when a file is migrated. By default, it will get the stripe_count and stripe_size of the file (maybe pool and mirror_count if <a href="https://review.whamcloud.com/32977" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32977</a> is updated to do so), but it does not preserve more complex layout details.</p> <p>Dropping the initial components of a PFL file is probably desirable, since it is unlikely that a file will become smaller, and it makes sense to keep the stripe_count of a file at the widest needed.</p> <p>We <b>do</b> need to keep the number of mirrors across migration, and it would probably be good to keep any OST pools per mirror, if they exist.</p> <p>Separately, it would be useful to have some way to migrate <b>only</b> the mirror(s) that exist on particular OSTs if we are trying to empty particular OSTs.</p> LU-11510

preserve PFL/FLR/DoM layout with lfs_migrate

Bug Minor Resolved Fixed Emoly Liu Andreas Dilger Fri, 12 Oct 2018 03:01:54 +0000 Mon, 22 Jun 2020 22:59:51 +0000 Thu, 5 Mar 2020 22:55:22 +0000 Lustre 2.11.0 Lustre 2.12.0 Lustre 2.14.0 Lustre 2.12.5 0 8 <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=adilger" class="user-hover" rel="adilger">adilger</a>,</p> <p>To meet the requirement, the following changes will be done to lfs_migrate tool:</p> <ul> <li>use yaml file to save migrated file layout</li> <li>save the detailed OST pool information</li> </ul> <p>What's more, do we need to add options to control these features or by default? </p> <p>I think the easiest way to implement this is to leverage the "<tt>lfs setstripe --copy</tt>" option to copy the layout from the source file to the target file. It may be that with PFL/FLR/DoM files that we should change lfs_migrate to not try to get the source stripe_count and stripe_size and just copy the layout from the source file (ie. do less handling of the layout rather than more)?</p> <p>Emoly Liu (emoly@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36082" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36082</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11510" title="preserve PFL/FLR/DoM layout with lfs_migrate" class="issue-link" data-issue-key="LU-11510"><del>LU-11510</del></a> scripts: add --copy and --yaml to lfs_migarate<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 16fb4f3eb169c05809784d14f54cf3fd4f67fd36</p> <p>It seems like the core problem is that "<tt>lfs migrate</tt>" doesn't handle PFL files very well. Starting with a 2-component stripe_count=&lt;1, 4&gt; file (with the 4-stripe component initialized):</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># lfs getstripe !$ lfs getstripe /myth/tmp/pfl/64M /myth/tmp/pfl/64M lcm_layout_gen: 3 lcm_entry_count: 2 lcme_id: 1 lcme_flags: init lcme_extent.e_start: 0 lcme_extent.e_end: 33554432 lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: 1 lmm_layout_gen: 0 lmm_stripe_offset: 2 lmm_pool: audio lmm_objects: - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x26cdc7:0x0] } lcme_id: 2 lcme_flags: init lcme_extent.e_start: 33554432 lcme_extent.e_end: EOF lmm_stripe_count: 4 lmm_stripe_size: 1048576 lmm_pattern: 1 lmm_layout_gen: 0 lmm_stripe_offset: 4 lmm_objects: - 0: { l_ost_idx: 4, l_fid: [0x100040000:0x311e78:0x0] } - 1: { l_ost_idx: 0, l_fid: [0x100000000:0x22ca87:0x0] } - 2: { l_ost_idx: 1, l_fid: [0x100010000:0x1b1931:0x0] } - 3: { l_ost_idx: 3, l_fid: [0x100030000:0x22233d:0x0] } </pre> </div></div> <p>Using "<tt>lfs getstripe -c /myth/tmp/pfl/64M</tt>" returns a stripe_count of <tt>4</tt> from the last instantiated component as one would expect. Running "<tt>lfs migrate</tt>" without any arguments doesn't preserve the PFL layout <b>AND</b> it doesn't even keep the stripe count from the last initialized component of the source file. At a minimum, "<tt>lfs migrate /myth/tmp/pfl/64M</tt>" should use <tt>stripe_count=4</tt> to create the new file, or copy the whole PFL layout but instead it is using the filesystem-wide default <tt>stripe_count=1</tt>:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># lfs getstripe !$ lfs getstripe /myth/tmp/pfl/64M /myth/tmp/pfl/64M lmm_stripe_count: 1 lmm_stripe_size: 1048576 lmm_pattern: 1 lmm_layout_gen: 4 lmm_stripe_offset: 3 obdidx objid objid group 3 2237247 0x22233f 0 </pre> </div></div> <p>I think it makes sense to use the "<tt>lfs migrate &#45;&#45;copy</tt>" option internally to copy the layout from the source file, or from the parent directory for "<tt>lfs_migrate &#45;R</tt>". The "<tt>&#45;&#45;yaml</tt>" option still can specify an arbitrary layout. That can potentially simplify migration </p> <p>Hi Emoly/Andreas, do we have any updates on this?</p> <p><del>Emoly Liu (emoly@whamcloud.com) uploaded a new patch:</del> <a href="https://review.whamcloud.com/37401" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37401</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-11510" title="preserve PFL/FLR/DoM layout with lfs_migrate" class="issue-link" data-issue-key="LU-11510"><del>LU-11510</del></a> lfs: create a comp file correctly when migrating</del><br/> <del>Project: fs/lustre-release</del><br/> <del>Branch: master</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: a12d86989152bc6fbb9ad3e6566c24c29b077f76</del></p> <blockquote> <p>the next step is to move the file to the new OSTs by adding an option to lfs_migrate tool to let the customer specify the OST list.</p></blockquote> <p>Normally when doing large-scale migration, the OST selection would be handled by the MDS by disabling object creation on some OSTs (via "lctl set_param osp.&lt;fsname&gt;-OST<span class="error">&#91;xxxx-yyyy&#93;</span>.max_create_count=0") and running "lfs find -ost xxxx,...,yyyy" to list all of the files on those OSTs to migrate. I think the common case is finding files on OSTs that <em>shouldn't</em> be used, and the MDS QOS policy will already bias file allocation onto OSTs with more free space.</p> <p>I think would be useful for making easier file layout policies for users are:</p> <ul class="alternate" type="square"> <li>using "<tt>lfs setstripe -<del>copy XXX</tt>" to select a template layout from an existing file/directory (this should already work with the new patch), but allowing an "alias" like <tt></del>-copy=@mypattern</tt> that looks for the file "<tt>$HOME/.lfs/mypattern</tt>" or similar</li> <li>finishing patch <a href="https://review.whamcloud.com/28972" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28972</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-9982" title="Clients striping from mapped FID in nodemap" class="issue-link" data-issue-key="LU-9982">LU-9982</a> lustre: Clients striping from mapped FID in nodemap</tt>" so that it is possible to set a per-fileset default layout</li> <li>implementing the <a href="https://jira.whamcloud.com/browse/LU-9392" title="lfs migrate -o and lfs setstripe -o should pick OST from ost_list" class="issue-link" data-issue-key="LU-9392">LU-9392</a> functionality to allow <tt>setstripe</tt> to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins</li> </ul> <blockquote> <p>implementing the <a href="https://jira.whamcloud.com/browse/LU-9392" title="lfs migrate -o and lfs setstripe -o should pick OST from ost_list" class="issue-link" data-issue-key="LU-9392">LU-9392</a> functionality to allow setstripe to specify a range of OSTs, but set the stripe count smaller than the OST list, similar to how an OST pool works, but available to users instead of admins</p></blockquote> <p>This is close to my idea.</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36082/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36082/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11510" title="preserve PFL/FLR/DoM layout with lfs_migrate" class="issue-link" data-issue-key="LU-11510"><del>LU-11510</del></a> lfs: migrate a composite layout file correctly<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 8bedfa377fbd0c9f1b6ea2c40d36fdcaa52137df</p> <p>Landed for 2.14</p> <p>Minh Diep (mdiep@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/37814" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37814</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11510" title="preserve PFL/FLR/DoM layout with lfs_migrate" class="issue-link" data-issue-key="LU-11510"><del>LU-11510</del></a> lfs: migrate a composite layout file correctly<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: fdb231d505dda018515f754475527041e67af7c8</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37814/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37814/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11510" title="preserve PFL/FLR/DoM layout with lfs_migrate" class="issue-link" data-issue-key="LU-11510"><del>LU-11510</del></a> lfs: migrate a composite layout file correctly<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: dc88cb0843e57be3930558e4e1b5c9b5d3af3311</p> Related LU-10801 LU-9982 LU-9392 LU-13703 LU-9629 Development Rank 1|i0043r: Rank (Obsolete) 9223372036854775807 Severity