[LU-4684] DNE3: allow migrating DNE striped directory

• Type: Improvement
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.12.0, Lustre 2.14.0
• Affects Version/s: Lustre 2.6.0, Lustre 2.7.0, Lustre 2.8.0
• Labels: dne3
• Assignee: Lai Siyao
• Reporter: Andreas Dilger

Description

The lfs migrate patch http://review.whamcloud.com/6662 will only allow migrating from a 1-stripe directory to another 1-stripe directory. That is unfortunate given that DNE2 is implementing striped directories. At a minimum this limitation needs to be documented for lfs migrate in the 2.6.0 release.

I think that implementing migration from one striped directory to another striped directory (restripe) is not very much more complex. Essentially, walk the source directory(ies), hash the name, migrate it to the correct target directory stripe. For migration, the "right" directory will never be the source directory, but for restriping a single directory to multiple, some of the entries will already be in the right place (stripe 0) and others would need to move to stripe N based on the hash. If changing from M to N stripes or changing the hash function, 1/M entries would already be on the right MDT and wouldn't need to be moved.

Issue Links

• is blocked by: LU-7357 Add layout lock for striped directories.
• is duplicated by: LU-11191 Striped DIR accross 2 MDTs very slow
• is duplicated by: LU-4029 DNE (distributed namespace) and legacy filesystems
• is related to: LU-4876 LFSCK remove entry from /REMOTE_PARENT_DIR if MDT-object name reside on the same MDT after migration
• is related to: LU-5550 LFSCK 5: Migrate the name entry from one MDT to another MDT if the name hash does not match the striped directory
• is related to: LU-11611 Incorrect return value in mdt_reint_unlink
• is related to: LU-6515 Always true check in mdd_migrate_create
• is related to: LU-11025 DNE3: directory restripe
• is related to: LU-10329 DNE3: REMOTE_PARENT_DIR scalability
• is related to: LUDOC-395 Add documentation for DNE striped directory migration
• is related to: LU-3531 DNE2: striped directory