[LU-2430] Migration tool for DNE

Related — Wed, 17 Apr 2019 17:15:23 +0000

Comments from Andreas:

This relates to the need to rebalance the inodes between an existing MDT and a newly-added MDT in some manner, preferably without having to move all of the file data just to make namespace changes.
We discussed something like "lfs mv -i

Unknown macro: {mdt_idx}

Unknown macro: {pathname}

" (same parameters as "lfs mkdir"/"lfs setdirstripe") to move the name and inode over to the specified MDT, but not keeping the same FID for that file. The inode number is not preserved for local filesystems when "mv" is called to move a file across different filesystems. This would only work for non-directories, since the ability to migrate a whole directory is difficult and does not need to be implemented for the first version of this tool.

Moving a whole directory hierarchy is non-trivial, since we don't want the namespace to be split if the user-space tool is interrupted in some way. One possible way to do this safely without the ability to migrate whole directories is:

do a breadth-first traversal of the directory tree
create a duplicate directory hierarchy on the target MDT using llapi_mkdir()
if the file is small, just copy it to the new directory
if the file is large hard link it into the new hierarchy (leave remote inode in place)
delete the old hierarchy (to get rid of the hard links/copies)
if the inode is remote with 1 hard link, call llapi_mv() to move the file name and inode (still resident on the original MDT) to the new MDT

This is not mandatory for DNE phase I, but nice to have. And this is definitely needed for DNE phase II.

Whamcloud Community JIRA

[LU-2430] Migration tool for DNE