https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11421 Lustre <p>Make migration for DoM files with LFS command. This is not working out-of-box for Data-on-MDT files because it is not enough just change layout, data should be moved as well.</p> <p>The OST-to-MDT and MDT-MDT migrations to be supported. Note that MDT-MDT migration might just be "cp + rename", since it will be the same.</p> LU-11421

DoM: manual migration OST-MDT, MDT-MDT

Technical task LU-10176 Major Resolved Fixed Mikhail Pershin Andreas Dilger DoM2 Mon, 24 Sep 2018 14:45:40 +0000 Wed, 27 Apr 2022 23:54:10 +0000 Mon, 16 Sep 2019 23:30:47 +0000 Lustre 2.13.0 0 6 <p>Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/35359" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35359</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11421" title="DoM: manual migration OST-MDT, MDT-MDT" class="issue-link" data-issue-key="LU-11421"><del>LU-11421</del></a> dom: manual OST-to-DOM migration via mirroring<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 95a57898961152c2d8dc6dc6e94399b0baffa81f</p> <p>Add here some explanation about how OST-DOM migration works:</p> <p>OST layout + data on OSTs -&gt; DOM layout + data on MDT</p> <ul class="alternate" type="square"> <li>no sense to create volatile file, because its data will be tied to its new MDT inode, but we want to keep data with the old inode. We need to read data from OSTs stripes and write it to the inode blocks on MDT (like to the DOM file)</li> <li>add new mirror with DOM component via 'mirror extend', the DOM component will be 'stale' after that</li> <li>do mirror resync to fill stale DOM component with data</li> <li>remove old layout from mirror by 'mirror split'</li> </ul> <p>It is possible to have a DOM component with the same size in different mirrors, it is not 'mirrored' in that case but since we are considering MDT inode exists always that is not a problem I think. Technically it can be different size even - MDT inode will store the largest DoM stripe in that case but more work will be needed to return MDT stripe size to the client correctly in that case. Now it is just inode size, but should be limited by DOM component size depending on chosen mirror layout.</p> <p>Another thing to think about - what sort of OST-striped files should NOT be migrated to DOM files:</p> <ul class="alternate" type="square"> <li>file size is bigger already than proposed DOM component size - can't imagine useful reason for such migration with no any benefits from MDT stripe</li> <li>already mirrored files - new DOM layout should be added manually as new mirror in that case, not via <tt>lfs migrate</tt></li> <li>not sure about PFL files, if file was created as PFL so it is expected it will grow beyond the current size, so also no sense to move it on MDT, on other hand excluding such files would mean that only plain layout files can be migrated to MDT via <tt>lfs migrate</tt></li> </ul> <blockquote> <p>Add here some explanation about how OST-DOM migration works:</p></blockquote> <p>This explanation should all be included in the patch commit message. </p> <p>Note that we can't exclude PFL files just on principle, because a filesystem may have a default PFL layout (maybe before the MDT has enough space for DoM, then new large MDTs are added to the filesystem) so <b>all</b> files are PFL. In general, while. It is good to have smart behavior by default if no other input is given, I think the kernel should try to avoid overriding decisions made by userspace. </p> <p>yes, I think that PFL files should be processed as all other, if it is file size bigger than DOM component size of new layout then <tt>lfs migrate</tt> should exit with a warning about that and propose to use <tt>-f</tt> parameter if user really want to do that.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>yes, I think that PFL files should be processed as all other, if it is file size bigger than DOM component size of new layout then lfs migrate should exit with a warning about that and propose to use -f parameter if user really want to do that. </pre> </div></div> <p>Well, it isn't clear that there is a need/benefit to return an error when the user asks for this. There <b>are</b> reasons for having a DoM component at the start of a file, e.g. for files that have an embedded index/icon/header at the start, so my first choice would be to allow what the user asked for instead of trying to second-guess their request. That said, if the user has <b>not</b> requested a DoM component (e.g. generic "<tt>lfs migrate</tt>" command) then I'm perfectly happy to drop the DoM component, and PFL in general, and use a plain layout with the <tt>stripe_count</tt>, <tt>stripe_size</tt>, and <tt>pool</tt> from the last instantiated component of the file.</p> <p>As an aside, I generally dislike using "<tt><del>f</tt>" as "force some action", since often the "<tt>-f</tt>" argument gets overloaded to mean "force common action A" and also "force dangerous action B", and people just get used to adding "-f" to all of their commands which can lead to bad things happening (e.g. "<tt>rm -rf / &#42;</tt>"). It would be better to have a long option like "<tt></del>-force-mdt</tt>", but in general I think if the user already asks for a DoM component via "<tt>-E 1M --layout mdt ...</tt>" then that is enough.</p> <p>Is it also possible to do DoM-to-OST mirroring to drop <em>just</em> the <tt>mdt</tt> component from a large file? That would essentially need to write the DoM data to the first OST object (second component) in the background, and then add a new xattr operation to drop the <tt>mdt</tt> component and change the start of the second component to offset 0.</p> <p>Andreas, yes, I tend to agree, though several DOM optimisations are lost for files with DOM+OST objects instantiated there are still some remains, e.g. small random access moved from OST to MDT and considering that MDT can have faster storage in general. So it is even simpler for me to don't add extra checks for <tt>lfs migrate</tt>.</p> <p>As for the DOM component removal, I think that has the similar benefits for any PFL file, when upon growing the first, smaller, component could be integrated into the next one and dropped. Though I am not sure if we have such feature now.</p> <p>Meanwhile, mirroring allows also to increase DoM component size for DoM files, which is not possible via layout swap.</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/35359/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35359/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11421" title="DoM: manual migration OST-MDT, MDT-MDT" class="issue-link" data-issue-key="LU-11421"><del>LU-11421</del></a> dom: manual OST-to-DOM migration via mirroring<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 44a721b8c10631b52f9ee2fbac1eee8cb775d148</p> <p>Landed for 2.13</p> <p>Is it planned to backport this patch to b2_12? I'm asking because we have MDTs that are almost full in terms of inodes (mainly due to the DoM ldiskfs space requirement, we ran out of inodes even though each MDT is 18TB). Many DoM files remain on these full MDTs, so we cannot easily migrate directories them to other MDTs (<a href="https://jira.whamcloud.com/browse/LU-13298" title="lfs migrate -m &quot;migrate failed: Operation not supported (-95)&quot; on DoM files" class="issue-link" data-issue-key="LU-13298"><del>LU-13298</del></a>).</p> Cloners LU-10177 Related LU-3285 LU-10258 LU-10176 LU-15794 LU-10910 LU-12935 LU-15219 LU-10112 LU-10995 LU-13612 Development Rank 1|i002yv: Rank (Obsolete) 9223372036854775807