https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-17107 Lustre <p>Hi,</p> <p>In our last data migration efforts involving 50 OSS servers, we observed slow migration speeds even though we used a lot of concurrent threads with the lfs_migrate command. To address this, we switched to the cp and mv methods later, which greatly enhanced our migration speed.</p> <p>The migration command we initially used is as below:</p> <p>lfs find --ost ost_name /mount/point | lfs_migrate -y</p> <p> </p> <p>As we prepare for our next migration task, I'd like to know if there are better migration methods we haven't considered yet.</p> <p>Thank you for your guidance,</p> <p>Jane</p> RHEL8 LU-17107

Seeking Advice on Efficient Data Migration Methods

Question/Request Minor Open Unresolved Colin Faber BNL Team Mon, 11 Sep 2023 14:23:08 +0000 Fri, 22 Sep 2023 21:38:59 +0000 Lustre 2.15.2 0 3 <p>Hi,</p> <p><a href="https://hpc.github.io/mpifileutils/" class="external-link" target="_blank" rel="nofollow noopener">https://hpc.github.io/mpifileutils/</a> might be a useful tool for this task and help speed up your migration dramatically.</p> <p>Hi,</p> <p>Thanks for the link. I checked mpifileutils but only saw that it can be used to migrate from one lustre to another lustre. What I need is a fast approach to migrate files from a bunch of retiring OSTs to other active OSTs. Is there a way to do it using mpifileutils? If yes, how to do it? </p> <p>Thanks,</p> <p>Jane</p> <p>If you have a specific layout defined, dsync from mpifileutils can be leveraged to copy into these targets.</p> Development Rank 1|i03v5z: Rank (Obsolete) 9223372036854775807 Severity