https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4684 Lustre <p>The <tt>lfs migrate</tt> patch <a href="http://review.whamcloud.com/6662" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/6662</a> will only allow migrating from a 1-stripe directory to another 1-stripe directory. That is unfortunate given that DNE2 is implementing striped directories. At a minimum this limitation needs to be documented for <tt>lfs migrate</tt> in the 2.6.0 release.</p> <p>I think that implementing migration from one striped directory to another striped directory (restripe) is not very much more complex. Essentially, walk the source directory(ies), hash the name, migrate it to the correct target directory stripe. For migration, the "right" directory will never be the source directory, but for restriping a single directory to multiple, some of the entries will already be in the right place (stripe 0) and others would need to move to stripe N based on the hash. If changing from M to N stripes or changing the hash function, 1/M entries would already be on the right MDT and wouldn't need to be moved.</p> LU-4684

DNE3: allow migrating DNE striped directory

Improvement Major Resolved Fixed Lai Siyao Andreas Dilger dne3 Fri, 28 Feb 2014 01:30:55 +0000 Mon, 16 May 2022 21:53:26 +0000 Fri, 1 Apr 2022 00:31:42 +0000 Lustre 2.6.0 Lustre 2.7.0 Lustre 2.8.0 Lustre 2.12.0 Lustre 2.14.0 0 8 <p>I probably can not fix this in 2.8, so move it to 2.9?</p> <p>Assign to Lai for now, you can work out with Fan Yong if he has more time for this one (after the ZFS OI Scrub is done?)</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/26297" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/26297</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> dne: support migrating striped directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 19779a48fe647aa1a420e396ce2720dc33219a4b</p> <p>Directory migration should be implemented very similar to directory restriping, so that they can share the same code. Essentially, create a new layout for the directory that includes old and new shards (directory stripes), then iterate over all of the entries and move them to the shard that they belong to in the new layout. This ensures that if the (long and not atomic) operation is interrupted in the middle all of the entries can still be found in either the old or new location (which is the reason for LMV_HASH_FLAG_MIGRATION - so that the client will try all shards to find the entry) since the entries will always remain in the one directory layout even during movement. It would even allow migration to happen while the directory is being accessed, since we only need to lock one entry exclusively at a time.</p> <p>With directory restriping, only the existing name entries (not inodes) would be moved to the new shards, and newly-created entries would be created on the correct MDTs. When moving from a non-striped directory to a striped directory, the non-striped directory would become the master entry, and it would gain new shards to hold all of the name entries. When moving from one stripe count to another (or changing hash functions), then new shards would be added at the start of the migration operation, all entries are migrated according to the new hash function/modulus, then if the stripe count is reduced then the (verified-to-be-empty) old shard(s) would be removed.</p> <p>With directory migration the hash function would need to be enhanced to add new shard(s) at the end and store an "offset" or similar parameter in the LMV EA, or otherwise flag the old shards as "do not use", and restripe from the source shards to the target shards.</p> <p>All of these operations would essentially be handled by introducing a new hash function/flags that properly maps entries from one set of shards to another. Either redistributing entries evenly across shards in the restriping case, or leaving one or more shards empty in the migration case.</p> <p>We might consider optimizing the case of removing only a single shard (e.g. MDT removal) during migration, where the new shard is added at the end, all entries are moved from the old shard to the new one, and then the new shard replaces the old one in the layout. During this time, the hash function should keep names mapped to the other existing shards unchanged so that they do not all need to be migrated. Otherwise, all name entries in this directory will become remote and hurt access performance.</p> <p>The inodes themselves would also need to be moved during migration (not necessarily restriping, if we do this only when there are a relatively small number of entries in the directory, say &lt; 32000), but inode migration is largely independent of how the directory layout causes name entries to be hashed. Essentially, the DNE code would be told whether only the name is being migrated, or if the whole inode needs to be migrated also. The name migration can happen in one step, then the inode migration can happen in a second step (either interleaved with name migration, or as a second step after all names have been moved.</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/29822" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29822</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> lod: store dir stripe fids in lmm<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 07af57e02f1dfaf502daa03fb26652a546dc4ed1</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31424" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31424</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> dne: pack lmv ea in migrate rpc<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 38179cf0f839d13196afbe2cf03a1d87d7d19098</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31425" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31425</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> mdt: improve directory stripe lock<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 1d060d3815496b9006aaa24d74e7f55f12c42292</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31426" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31426</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> xattr: add list support for remote object<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: aa371d40fe401cbffaf3829ba6d3c72162214528</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31427" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31427</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> dne: migrate striped directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 4b39b37e9e0cb2ce9b4f0653f1658f271066f2ae</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31504" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31504</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> lmv: support accessing migrating directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 39b623fb03ee32fa416b63707e2236aa063dbf39</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31626" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31626</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: shrink dir layout after migration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 299e2f67b285fbec41c8d6745a0cf8a32af5690e</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31914" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31914</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> ptlrpc: add dir migration connect flag<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 89459fc76b140ec6910a7bf46f1401a7c06c13dc</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/31914/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31914/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> ptlrpc: add dir migration connect flag<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 14b98596fa2433b6c8a8cce3ee5d0ddefab5982e</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32356" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32356</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> test: improve dir migrate test in racer<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 2a1e530bb5a03bd17a2899ca824060d7799b1a50</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32445" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32445</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> debug: dump objs in mdt_fini()<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 7af336e896416ee0a6492fe8fadab31998578538</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/31425/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31425/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> mdt: improve directory stripe lock<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 18aee6838907192c03c5f70e88624686c1c074da</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/31426/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31426/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> xattr: add list support for remote object<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 4325b1e456647f519a0eca32204554e0c358646f</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32589" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32589</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> lod: missing lock in lod_index_try()<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 81e60c65f7ed8b4faef2a56ef65657b1a340b3de</p> <p>Lai Siyao (lai.siyao@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32701" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32701</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> mdt: rename may cause deadlock<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: eff8d5bc1be10418fbcb056cef4bdc2f993d5285</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/32946" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32946</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> llite: add lock for dir layout data<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 3af345d83520bc9e485f44d1e6c00262df700c4e</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/32947" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32947</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: small fixes<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a0a8e19de507d9ee90d0b6c0798e9014d8e1368a</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/31424/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31424/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: pack lmv ea in migrate rpc<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 470bdeec6ca5b4c68f456a10d68511653e67b378</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/31427/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31427/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: migrate striped directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 169738e30a7e0b57f27a517d78d2c928b3bb0f5c</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/31626/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31626/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: shrink dir layout after migration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 0a83d948f37bec7fca6e9aa30f59f26354273b23</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/31504/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31504/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> lmv: support accessing migrating directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 976b609abcdf5adc29f55b742ff6b1307b2b6484</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33324" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33324</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: replace PFID via source<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 9a42c69380a905c7b9aad85dc70f9afad0ec2ac6</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33325" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33325</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: link parents lock may deadlock<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a410f20e682267e20e30eaab6aee299548322837</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/33456" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33456</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> lod: parse layout for migrating directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 515953a6ece1deb01720e63145e07e62add5f553</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33324/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33324/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> migrate: replace PFID via source<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: bd596fbe449f4fbab18ed184ccce1e141928b116</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33456/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33456/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> lod: parse layout for migrating directory<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 83aa8ffcf2e8a5b7da9ead941ed0b22a435d6790</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/32946/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32946/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> llite: add lock for dir layout data<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: ae828cd3b092a38adbc86a4da320dd9d3a0fc80c</p> <p>Landed for 2.12</p> <p>This ticket is still listed as the reason <tt>racer.sh</tt> is skipping directory migration.</p> <p>Either directory migration is working and can be enabled in <tt>racer.sh</tt> and remove <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> from the comment, or a new ticket should be filed (and marked <tt>always_except</tt>) with the issue(s) and a patch submitted to change <tt>racer.sh</tt> to list that ticket as the reason it is disabled and the <tt>always_except</tt> label removed from this ticket.</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/41359" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41359</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> tests: enable racer directory migration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 51f4f706aa94538af7d1703a4e46b47579b0cad6</p> <p><del>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch:</del> <a href="https://review.whamcloud.com/41360" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41360</a><br/> <del>Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> tests: enable racer directory migration</del><br/> <del>Project: fs/lustre-release</del><br/> <del>Branch: master</del><br/> <del>Current Patch Set: 1</del><br/> <del>Commit: 1c412393a42d454aa949fb0d87e788056d771c75</del></p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/41359/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41359/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-4684" title="DNE3: allow migrating DNE striped directory" class="issue-link" data-issue-key="LU-4684"><del>LU-4684</del></a> tests: enable racer directory migration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3070ca9b18206025d9fd55817bf4da1ec486b6be</p> Blocker LU-7357 Duplicate LU-11191 LU-4029 Related LU-4876 LU-5550 LU-11611 LU-6515 LU-11025 LU-10329 LUDOC-395 LU-3531 Development Rank 1|hzwga7: Rank (Obsolete) 12872