[LU-8207] Add auto-stripe option to lfs_migrate

Related — Fri, 3 Jan 2020 23:59:07 +0000

The primary reason for this feature is to be able to give users a tool to fix stripe settings on existing files based on file size. Optimal stripe selection for performance is dependent on access patern, but we can make the assumption that large files are more likely to be accessed by multiple clients and benefit from wider striping. The other benefit from restriping is to balance OST capacity used, which an algorithm based solely on file size can more easily address.

The goal is to have a tool to give the users and say "go restripe your directory with this command" and it will do the right thing in 90% of cases. lfs_migrate already has most of the needed functionality. The change would be to add a "-A" flag to automatically select the stripe count as the file is rewritten. Initial algorithm to determine stripe count is Log2(size_in_GB)+1, though this could change in the future.

The other application of this patch would be to integrate it with Robinhood (or "lfs find") to generate a list of large, inactive, and narrowly-striped files and pass them to lfs_migrate for restriping. This could be coupled well with OST rebalancing actions.

For further discussion and context, see mailing list thread "stripe count recommendation, and proposal for auto-stripe tool":
http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/2016-May/013526.html

Whamcloud Community JIRA

[LU-8207] Add auto-stripe option to lfs_migrate