[LU-14211] DNE3: mechanism to interrupt and resume migration

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Labels: dne3
• Assignee: WC Triage
• Reporter: Andreas Dilger

Description

It should be possible to cleanly interrupt DNE directory migration (e.g. at the end of the current directory) for a long-running recursive directory migration.

It would make sense to restructure recursive directory migration as a series of single directory migrations now that this is possible (LU-14975). This would provide a number of benefits:

• allow the migration to be interrupted after the current directory has finished
• allow statistics printing to be done by "lfs migrate -m" in userspace after each directory finishes
• avoid very long-running processes on the MDS
• simplify restart of directory tree migration
• allow better (automated) per-directory tuning (stripe count 1/N for small/large directories, select different target MDT by space) as each directory is migrated, instead of using the same parameters for all directories in the tree

Issue Links

• is related to: LU-14212 DNE3: directory migration progress monitor
• is related to: LU-11776 add "lfs find" support for directory hash flags
• is related to: LU-15990 "lfs find" to scan for directory hash flags
• is related to: LU-14719 "lfs migrate -m" creates broken agent inodes when target MDT full
• is related to: LU-14975 DNE3: directory migration in non-recursive mode
• is related to: LU-15001 improve recovery of interrupted directory migrate