[LU-16948] sparse file migration broken Created: 06/Jul/23 Updated: 02/Aug/23 |
|
| Status: | Open |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.16.0, Lustre 2.15.3 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor |
| Reporter: | Etienne Aujames | Assignee: | Etienne Aujames |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Issue Links: |
|
||||||||
| Severity: | 3 | ||||||||
| Rank (Obsolete): | 9223372036854775807 | ||||||||
| Description |
|
On 2.15.3 sparse file migration does not keep the same number of blocks if the extents are non-aligned on the stripe_size (on master the extent must be > 64M): Reproducer [root@client client]# lfs setstripe -c2 test_simple
[root@client client]# dd if=/dev/urandom of=test_simple bs=512K count=$((128*2+1)) seek=1
[root@client client]# dd if=/dev/urandom of=test_simple bs=512K count=$((128*2+1)) seek=$((512*2+1)) conv=fsync
[root@client client]# filefrag -v test_simple
Filesystem type is: bd00bd0
File size of test_simple is 672137216 (656384 blocks of 1024 bytes)
ext: device_logical: physical_offset: length: dev: flags:
0: 0.. 3: 4063232.. 4063235: 4: 0001: net
1: 512.. 66559: 4063744.. 4129791: 66048: 0001: net
2: 262656.. 319487: 4137472.. 4194303: 56832: 0001: net
3: 319488.. 327679: 466944.. 475135: 8192: 0001: net
4: 327680.. 328703: 491520.. 492543: 1024: 0001: net
5: 0.. 65535: 2883584.. 2949119: 65536: 0000: net
6: 262144.. 327679: 2949120.. 3014655: 65536: 0000: last,net
test_simple: 5 extents found
[root@client client]# du test_simple
263176 test_simple
[root@client client]# strace -o report lfs migrate -c2 test_simple
[root@client client]# filefrag -v test_simple
Filesystem type is: bd00bd0
File size of test_simple is 672137216 (656384 blocks of 1024 bytes)
ext: device_logical: physical_offset: length: dev: flags:
0: 0.. 3: 433676.. 433679: 4: 0001: net
1: 512.. 1023: 457244.. 457755: 512: 0001: net
2: 1024.. 1535: 461812.. 462323: 512: 0001: net
3: 1536.. 2047: 457756.. 458267: 512: 0001: net
4: 2048.. 3583: 462848.. 464383: 1536: 0001: net
5: 3584.. 4095: 397824.. 398335: 512: 0001: net
6: 4096.. 24575: 499712.. 520191: 20480: 0001: net
7: 24576.. 66559: 2629632.. 2671615: 41984: 0001: net
8: 262656.. 328703: 2679296.. 2745343: 66048: 0001: net
9: 0.. 1023: 473088.. 474111: 1024: 0000: net
10: 1024.. 1535: 466432.. 466943: 512: 0000: net
11: 1536.. 2047: 475136.. 475647: 512: 0000: net
12: 2048.. 3071: 474112.. 475135: 1024: 0000: net
13: 3072.. 3583: 475648.. 476159: 512: 0000: net
14: 3584.. 66047: 3014656.. 3077119: 62464: 0000: net
15: 262144.. 327679: 3088384.. 3153919: 65536: 0000: last,net
test_simple: 16 extents found
[root@client client]# du test_simple
263692 test_simple
block_diff = 263692 - 263176 = 516kb "lfs migrate" strace: lseek(3, 4096, SEEK_DATA) = 524288 lseek(3, 524288, SEEK_HOLE) = 135266304 pread64(3, "]\220\311\337\376 T@\3622\314\231\3\0100\202]\31j\246u\177\256\255\307\250\0\333\311\301\346\251"..., 1048576, 524288) = 1048576 pwrite64(4, "]\220\311\337\376 T@\3622\314\231\3\0100\202]\31j\246u\177\256\255\307\250\0\333\311\301\346\251"..., 1048576, 524288) = 1048576 ... pwrite64(4, "\266\270\20\357\243\366#jC\217\333\325\260V\314;\274b\204\250\4s\\\231\6\222\34\300\30n]\27"..., 1048576, 133693440) = 1048576 pread64(3, "\325/\377m\316\251P\2723\27*%1\205\216\365; \tv$\241X\31>\342\331f\265td\353"..., 1048576, 134742016) = 1048576 pwrite64(4, "\325/\377m\316\251"..., 1048576, 134742016) = 1048576 <------ 134742016 + 1048576 = 135790592 != seek_hole (135266304) lseek(3, 135790592, SEEK_DATA) = 537395200 lseek(3, 537395200, SEEK_HOLE) = 672137216 |
| Comments |
| Comment by Patrick Farrell [ 06/Jul/23 ] |
|
"On 2.15.3 sparse file migration with a stripe count > 2 does not keep the same number of blocks if the extents are non-aligned on the stripe_size (on master the extent must be > 64M):" Etienne, I don't follow the second part here - What is the bug on master exactly? Also, did you mean stripe count > 1 ? |
| Comment by Etienne Aujames [ 06/Jul/23 ] |
|
Sorry, my mistake, the stripe_count is not the culprit (I can reproduce this with stripe_count=1). |
| Comment by Gerrit Updater [ 06/Jul/23 ] |
|
"Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/51593 |
| Comment by Etienne Aujames [ 06/Jul/23 ] |
|
This bug is linked to the buffer_size, on master the buffer_size is > 64M: |
| Comment by Etienne Aujames [ 06/Jul/23 ] |
|
On master the block_diff should be something like this: |