[LU-13397] lfs migrate/mirror extend/resync does not preserve sparse file Created: 28/Mar/20 Updated: 17/Mar/23 Resolved: 26/Jan/22 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | Lustre 2.15.0 |
| Type: | Bug | Priority: | Major |
| Reporter: | Andreas Dilger | Assignee: | Mikhail Pershin |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | HPv3, HPv4p1 | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Severity: | 3 | ||||||||||||||||||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||||||||||||||||||
| Epic Link: | Hot Pools - v4 phase 1 | ||||||||||||||||||||||||
| Description |
|
While testing "lfs migrate", "lfs mirror extend", and "lfs mirror resync", I was doing high-offset writes to initialize later PFL components of the file layout (e.g. at 1GB offset and 32GB offset). When trying to mirror/resync those files to another mirror copy, it resulted in the commands failing due to ENOSPC because there was not enough free space in the test filesystem to write out the data. These commands should be updated to share a common "data copy" routine (if they don't already) to reduce code duplication when fixing these issues. Then, the code needs to handle sparse input files properly, first by checking for sparse files (e.g. blocks << size) to enable checking the source file, and not copying holes in the file. There should probably be options added for each command like "--sparse=<auto,no,yes>" (default = auto) to force a specific behavior. Unfortunately, there is no optimal way to handle reading of sparse files in Lustre today. In all cases, it makes little sense to be doing these operations on in-use files, so there are already checks if the file is modified during migrate/mirror/resync.
|
| Comments |
| Comment by Andreas Dilger [ 28/Mar/20 ] |
|
Another option here (that could also be very useful for other reasons) might be to allow specifying "--copy-cmd" for these tools to offload the handling of the data movement from "lfs". The "lfs migrate/mirror" command would handle creating the file layout and opening file handles for the source and destination, and then execute "copy-cmd %s %s", e.g. "cp --sparse=auto $src $tgt" or "dcp $src $tgt". It would be possible for "lfs migrate" and "lfs mirror extend" to potentially use a copy command that only worked on filenames instead of file handles, and then use layout swap or layout merge with a temporary victim file (the victim file could be volatile and accessed via $MOUNT/.lustre/fid/$FID). This is not necessarily possible for "lfs mirror resync" which needs to use O_DIRECT file handles to avoid cache issues on the file. It might be possible to pass the open handles to tools via "/dev/stdin" and "/dev/stdout" as the pathnames, but that isn't clear it would work for all cases (e.g. tool running on another node), but it wouldn't have to support every combination of uses. Alternately, a pathname like "$MOUNT/.lustre/fid/[fid_seq:fid_oid:0].comp_id" could be used to open a specific component of the file if the VFS treated the different open-by-FID paths as different inodes? Just an idea. |
| Comment by Gerrit Updater [ 03/Nov/20 ] |
|
Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/40530 |
| Comment by Gerrit Updater [ 26/Nov/20 ] |
|
Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/40772 |
| Comment by Gerrit Updater [ 26/Nov/20 ] |
|
Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/40773 |
| Comment by Li Xi [ 03/Mar/21 ] |
|
" |
| Comment by Gerrit Updater [ 30/Mar/21 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/40772/ |
| Comment by Gerrit Updater [ 06/Apr/21 ] |
|
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/40773/ |
| Comment by Mikhail Pershin [ 17/Aug/21 ] |
|
I've found the problem with mirror resync and sparse file, it is not working as intended, so I am keeping ticket opened. The problem is in fallocate punch operation during resync. If it is done prior write operation then write itself fails due to changed layout version. |
| Comment by Gerrit Updater [ 22/Aug/21 ] |
|
"Mike Pershin <mpershin@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/44721 |
| Comment by Gerrit Updater [ 22/Sep/21 ] |
|
"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/44721/ |
| Comment by Gerrit Updater [ 17/Mar/23 ] |
|
"Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50329 |