Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-2430

Migration tool for DNE

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Lustre 2.6.0
    • Lustre 2.6.0
    • None
    • 3
    • 5755

    Description

      Comments from Andreas:

      This relates to the need to rebalance the inodes between an existing MDT and a newly-added MDT in some manner, preferably without having to move all of the file data just to make namespace changes.
      We discussed something like "lfs mv -i

      Unknown macro: {mdt_idx}

      Unknown macro: {pathname}

      " (same parameters as "lfs mkdir"/"lfs setdirstripe") to move the name and inode over to the specified MDT, but not keeping the same FID for that file. The inode number is not preserved for local filesystems when "mv" is called to move a file across different filesystems. This would only work for non-directories, since the ability to migrate a whole directory is difficult and does not need to be implemented for the first version of this tool.

      Moving a whole directory hierarchy is non-trivial, since we don't want the namespace to be split if the user-space tool is interrupted in some way. One possible way to do this safely without the ability to migrate whole directories is:

      do a breadth-first traversal of the directory tree
      create a duplicate directory hierarchy on the target MDT using llapi_mkdir()
      if the file is small, just copy it to the new directory
      if the file is large hard link it into the new hierarchy (leave remote inode in place)
      delete the old hierarchy (to get rid of the hard links/copies)
      if the inode is remote with 1 hard link, call llapi_mv() to move the file name and inode (still resident on the original MDT) to the new MDT
      

      This is not mandatory for DNE phase I, but nice to have. And this is definitely needed for DNE phase II.

      Attachments

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              di.wang Di Wang
              di.wang Di Wang
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: