[LU-16446] 'lfs mirror extend' should allow specifying total mirror count

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.16.0
• Type: Improvement
• Priority: Minor
• Reporter: Andreas Dilger
• Assignee: WC Triage
• Resolution: Unresolved
• Labels: FLR, lad23dd, lfs, lug23dd, medium

Description

The "lfs mirror extend" command with the "-N" option today takes a count of the number of mirrors to add to a file, but there is no way to specify the total number of mirrors to create on a file. This can be problematic if there are multiple threads running e.g. "lfs mirror extend -N 1" on the same file, or if the command has an error and is retried, as it may incorrectly result in 3 or more mirrors being created on the file.

It would be desirable to have an option that specifies to create the total number of mirrors on the file, and this is a no-op if that number of mirrors already exists. It is open for discussion whether this should also reduce the number of mirrors on a file if there are more, and if so how it would select which mirror to remove.

Issue Links

• is related to: LU-5170 lfs usability
• is related to: LU-14554 Improve usability of "lfs mirror" com...
• is related to: LU-13403 ‘lfs mirror extend’ should not requir...
• is related to: LU-11022 FLR1.5: "lfs mirror" usability for Bu...

Comments

[Andreas Dilger]

It looks like it would be possible to use "lfs mirror extend -N =COUNT SETSTRIPE_OPTS FILE" or "lfs mirror extend --mirror-count==COUNT SETSTRIPE_OPTS FILE" to specify having exactly COUNT mirrors on a file. It should be possible to run this on files that already have COUNT mirrors and it would be a no-op.

I'm uncertain whether -N COUNT could/should remove excess mirrors already existing on a file.

• it would be convenient if there was some simple way to make files "consistent" regardless of whether they currently have more or fewer mirrors
• however, the command is named "mirror extend" so it might be confusing to users if it is reducing the number of mirrors
• it also isn't clear how this would select mirror(s) to remove if there are additional ones existing? One option might be to remove only additional mirrors that match the SETSTRIPE_OPTS specified on the command-line, with the assumption that those are the "extra" mirrors

It would be desirable to have a command to use for filesystem maintenance to run against a whole directory tree to bring all of the files to a consistent state. The problem of "lfs mirror extend -N COUNT" creating excess mirrors is somewhat "self healing" over time as users/tools move to use "-N =COUNT" to create mirrors, and files with excess mirrors are eventually deleted. However, it might be better to have a different command like "lfs mirror delete -N =COUNT" or similar that would reduce the number of mirrors on a file to exactly COUNT, along with a way to specify which mirrors to remove.

For example, "lfs mirror delete -N =2" would leave a file with exactly two mirrors, if the mirrors are "symmetrical" and it doesn't matter which one(s) are kept (maybe with some smarts to ensure the two remaining mirrors do not have components on overlapping OSTs). Something like "lfs mirror delete -N =1 --pool ssd -N =1 --pool hdd" could leave exactly one mirror on each of the ssd and hdd pools. This is just an initial idea at this point, and open for discussion for the best way to be able to specify such operations.