[LU-9961] FLR2: Relocating individual objects to a new OST

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.7.0, Lustre 2.9.0
• Labels: flr-improvement medium

Environment

Servers are currently CentOS-6.8 with Lustre-2.7.3 +NAS patches, planning to upgrade to CentOS-7.x and Lustre-2.10.x within a month or two.

Clients are currently SLES-12sp2 running Lustre-2.9.0 +NAS patches, planning to go to Lustre-2.10.x ASAP.

Filesystem is one of several, with multiple PB of capacity and 16 OSS nodes.

• Assignee: Peter Jones
• Reporter: Nathan Dauchy (Inactive)

Description

This ticket is twofold: 1) how do we efficiently migrate data off a set of OSTs with current or quick-hack tools? 2) what features and tools can be developed in the future to improve usability?

We have an older file system from which we are trying to remove a set of OSTs on a single server. To do this, we need to drain data off of the targets and onto the rest of the same file system. Initially we used the "lctl deactivate" method, but then switched to "lctl set_param osp.<OST>.max_create_count=0" due to the issues discussed in LU-4825 (and others). Unfortunately, users tend to keep data on this file system for a long time, so normal turnover is not significantly decreasing the usage of the OSTs.

Our understanding is that (with current versions of Lustre) to move objects off of the desired targets all parts of the file would have to be read and rewritten by a client, even if only one of many stripes is on the target. This is unacceptably inefficient, as the existence of files striped with a count of more than one results in a huge increase in the total data to be rewritten. In our case it would be well over half of the data on the file system. We are able to gather file size and stripe data (and the lists of files) using e2scan, Lester, and wrapper scripts.

1) What tools and methods are available to drain the OSTs more efficiently? Preferably it would involve a background process to copy just the individual objects on the OSS to be drained. And then we are open to taking a moderate downtime to just update the MDT to change layout info to point to the new object locations.

2) Can this use case be supported in the future by a tool like lfs_migrate, with options or auto-detection, to only move the parts of the file that need to be moved to satisfy the new target requirements? As an interim step, a server-side administrator command to initiate the object relocation would be fine.

Thanks!

 

Issue Links

• is related to: LU-12649 Tracker for ongoing FLR improvements
• is related to: LU-10911 FLR2: Erasure coding
• is related to: LU-12507 llapi: llapi_layout_pool_name_set() should "nuke" the ost objects in the layout structure
• is related to: LU-11621 Add copy_file_range() API and use it for lfs migrate and mirror resync
• is related to: LU-15424 allow 'lfs migrate' for only a single FLR mirror
• is related to: LU-17143 Use migrate_copy_data() for all data movement