Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16499

merge "lfs_migrate" functionality into "lfs migrate"

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • 9223372036854775807

    Description

      Having both the "lfs_migrate" script and "lfs migrate" command is confusing for users since they provide overlapping, but also slightly different functionality.

      "lfs_migrate" can be run as a pipe recipient like "xargs" and process filenames read from stdin. It isn't clear if that is strictly required, or if "| xargs lfs migrate ..." could be used as a replacement. One benefit of having a single "lfs_migrate" instantiation for the whole migration is that this would potentially allow better progress stats to be printed (LU-13482).

      "-rsync" will fall back to rsync to copy a file (or other command, LU-15613) if "lfs migrate" fails. This automatic fallback is deprecated in LU-13475. "-no-rsync" would be a no-op, if it is even offered, or just filtered out by a simple "lfs_migrate" wrapper.

      "-A" will "auto-stripe" a file based on the file size. This might be somewhat redundant now that PFL layouts are available, but can be useful in some cases. This short option does not conflict with anything in "lfs migrate" or "lfs setstripe", and should use "--auto|--auto-stripe" for the long option.

      "C <cap>" when -A is set, limit the migrated file to use on each OST at most 1/<cap> of the available space of the smallest OST. This conflicts with "lfs migrate -C|-overstripe-count" and would need to be changed, if still considered useful.

      "-M <min_free>" when -A is set, an OST must contain more available space than <min_free> KB in order for it to be considered available for use in the migration. This short option does not conflict with anything, and could use "--min-free" for the long option.

      "-q" quiets the printing of filenames and should be easily implemented.

      "-R" restripes the file based on the parent layout. This short option does not conflict with anything, and can use "--restripe" for the long option.

      "-X <max_free>" when -A is set, limit the amount of space on each OST that can be considered available for the migration to <max_free> KB. This conflicts with "-X" in "lfs setdirstripe" but nothing in "lfs setstripe" or "lfs migrate", and I don't think the conflict is critical.  It should use "--max-free" for the long option.

      "b|block|-non-block" already map to the same option in "lfs migrate"

      "D|-non-direct" already map to the same option in "lfs migrate"

      "-yaml|-file" copy the layout from the referenced file/dir, or read from the input file, and match the same options in "lfs migrate".

      "-0" (zero) input file names on stdin are separated by a NUL character instead of newline. This could be used to decide whether to read filenames from stdin instead of argv[].

      Attachments

        Issue Links

          Activity

            People

              wc-triage WC Triage
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: