[LU-16499] merge "lfs_migrate" functionality into "lfs migrate"

• Status: Open
• Project: Lustre
• Type: Improvement
• Priority: Minor
• Reporter: Andreas Dilger
• Assignee: WC Triage
• Resolution: Unresolved
• Labels: medium

Description

Having both the "lfs_migrate" script and "lfs migrate" command is confusing for users since they provide slightly different functionality.

"lfs_migrate" can be run as a pipe recipient like "xargs" and process filenames read from stdin. It isn't clear if that is strictly required, or if "| xargs lfs migrate ..." could be used as a replacement. One benefit of having a single "lfs_migrate" instantiation for the whole migration is that this would potentially allow better progress stats to be printed (LU-13482).

"-rsync" will fall back to rsync to copy a file (or other command, LU-15613) if "lfs migrate" fails. This automatic fallback is deprecated in LU-13475. "-no-rsync" would be a no-op, if it is even offered, or just filtered out by a simple "lfs_migrate" wrapper.

"-A" will "auto-stripe" a file based on the file size. This might be somewhat redundant now that PFL layouts are available, but can be useful in some cases. This short option does not conflict with anything in "lfs migrate" or "lfs setstripe".

"C <cap>" when -A is set, limit the migrated file to use on each OST at most 1/<cap> of the available space of the smallest OST. This conflicts with "lfs migrate -C|-overstripe-count" and would need to be changed, if still considered useful.

"-M <min_free>" when -A is set, an OST must contain more available space than <min_free> KB in order for it to be considered available for use in the migration. This short option does not conflict with anything

"-q" quiets the printing of filenames and should be easily implemented.

"-R" restripes the file based on the parent layout. This short option does not conflict with anything.

"-X <max_free>" when -A is set, limit the amount of space on each OST that can be considered available for the migration to <max_free> KB. This conflicts with "-X" in "lfs setdirstripe" but nothing in "lfs setstripe" or "lfs migrate", and I don't think the conflict is critical.

"b|block|-non-block" already map to the same option in "lfs migrate"

"D|-non-direct" already map to the same option in "lfs migrate"

"-yaml|-file" copy the layout from the referenced file/dir, or read from the input file, and match the same options in "lfs migrate".

"-0" (zero) input file names on stdin are separated by a NUL character instead of newline. This could be used to decide whether to read filenames from stdin instead of argv[].

Issue Links

• is related to: LU-15613 allow lfs_migrate to use 'cp' instead...
• is related to: LU-13482 add verbose stats to lfs_migrate and ...