https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10177 Lustre <p>Make migration for DoM files with LFS command. This is not working out-of-box for Data-on-MDT files because it is not enough just change layout, data should be moved as well.</p> LU-10177

DoM: manual migration MDT-OST

Technical task LU-10176 Major Resolved Fixed Mikhail Pershin Mikhail Pershin DoM2 Fri, 9 Dec 2016 09:26:39 +0000 Mon, 11 Feb 2019 17:38:44 +0000 Sat, 6 Oct 2018 13:18:25 +0000 Lustre 2.12.0 0 7 <p>1) MDT to OST migration<br/> DOM layout + data on MDT -&gt; OST layout + data on OST</p> <ul class="alternate" type="square"> <li>create volatile file (OST new layout);</li> <li>move data from DOM file to volatile file;</li> <li>swap layout (old DOM inode + new layout on OST);</li> <li>delete data on MDT by truncating local inode on MDT (*** new step). It is not trivial case too, need special code path.</li> </ul> <p>2) OST to MDT<br/> OST layout + data on OSTs -&gt; DOM layout + data on MDT</p> <ul class="alternate" type="square"> <li>no sense to create volatile file, because its data will be tied to its new MDT inode, but we want to keep data with the old inode. We need to read data from OSTs stripes and write it to the inode blocks on MDT (like to the DOM file)</li> <li>set DOM pattern to the original file</li> <li>create volatile file with the same OST layout (replica)</li> <li>copy data from it to the old file with DOM pattern</li> <li>delete stripes from the original file layout</li> <li>delete replica</li> </ul> <p>This should be able to use the existing FLR code paths on the client to open the MDT and OST components of the file for read/write to write <em>under</em> the other component. For some time at least this would become an FLR mirrored file, which is desirable to allow in any case. For FLR the DoM component would be preferred if the file size is below MDT size limit, and the OST component would be preferred if the file size is larger.</p> <p>Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/32979" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32979</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10177" title="DoM: manual migration MDT-OST" class="issue-link" data-issue-key="LU-10177"><del>LU-10177</del></a> lfs: support DOM-to-OST migration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: ce9af3080a1cc66eacb647940aa9b3502641c19c</p> <p>How much work will it be to be able to do the reverse - to be able to migrate small files to the MDT? I'm thinking that the FLR resync ioctl to open the MDT component to "write behind" the OST object and then just drop the OST component. </p> <p>well, my idea was to make that manually first with steps like:</p> <ul class="alternate" type="square"> <li>lfs mirror extend -N &lt;new DOM layout&gt; &lt;some non-dom file&gt;</li> <li>lfs resync</li> <li>lfs split --mirror-id 1 -d</li> </ul> <p>and make it work, it is not working just out of box and requires code update to make DOM code understand FLR cases, e.g. DOM component can be not first one with FLR, but still should be first one inside a mirror, also we need to allow only one DOM component in all mirrors and so on.<br/> I think that will be good as first step, at least simple script will be able to migrate files to MDT like that and Lustre code will be ready for FLR+DOM configurations. I'd say it shouldn't take a lot of time, but let me do more checks first.</p> <p>Next step will be implementing that in code as single call, probably with some optimizations.</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/32979/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32979/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10177" title="DoM: manual migration MDT-OST" class="issue-link" data-issue-key="LU-10177"><del>LU-10177</del></a> lfs: support DOM-to-OST migration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: f651470c13f83bc11237e12f6fdce9cdbf96561b</p> <p>Landed for 2.12</p> Cloners LU-11421 Related LU-11508 LU-3285 LU-10258 LU-10910 Development Rank 1|hzyy1z: Rank (Obsolete) 9223372036854775807