[LU-14212] DNE3: directory migration progress monitor

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Labels: dne3
• Epic Link: MDT rebalance v3
• Assignee: WC Triage
• Reporter: Andreas Dilger

Description

There should be some way to monitor the progress of a long-running recursive directory migration on the MDS.

Recursive directory migration should be replaced in userspace with multiple single-directory migrations (LU-14795) so that would be possible to have "lfs migrate -m" report progress in userspace after each directory is migrated, similar to file migration statistics in LU-13482 (the read/write bandwidth and total data migrated are not useful for directory migration):

directory:
- { seconds: <seconds>, files: <count>, fps: <files_per_sec> }
- { tot_sec: <seconds>, tot_files: <count>, avg_fps: <files_per_sec> }

seconds is the time taken to migrate the previous directory, files: is the number of files (all entries) in that directroy, fps=files/seconds for that directory. tot_sec: is the total elapsed time for the current migration, tot_files the total number of files (all entries) migrated so far, and avg_fps = tot_files/tot_sec for all entries migrated so far. The stats line(s) would only be shown every stats_interval seconds.

Issue Links

• is related to: LU-14466 metadata performance slows if the metadata migration is process is running
• is related to: LU-13482 add verbose stats to lfs_migrate and "lfs migrate"
• is related to: LU-14795 NVidia GDS support in lustre
• is related to: LU-14211 DNE3: mechanism to interrupt and resume migration