https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16948 Lustre <p>On 2.15.3 sparse file migration does not keep the same number of blocks if the extents are non-aligned on the stripe_size (on master the extent must be &gt; 64M):</p> <p><b>Reproducer</b></p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@client client]# lfs setstripe -c2 test_simple [root@client client]# dd if=/dev/urandom of=test_simple bs=512K count=$((128*2+1)) seek=1 [root@client client]# dd if=/dev/urandom of=test_simple bs=512K count=$((128*2+1)) seek=$((512*2+1)) conv=fsync [root@client client]# filefrag -v test_simple Filesystem type is: bd00bd0 File size of test_simple is 672137216 (656384 blocks of 1024 bytes) ext: device_logical: physical_offset: length: dev: flags: 0: 0.. 3: 4063232.. 4063235: 4: 0001: net 1: 512.. 66559: 4063744.. 4129791: 66048: 0001: net 2: 262656.. 319487: 4137472.. 4194303: 56832: 0001: net 3: 319488.. 327679: 466944.. 475135: 8192: 0001: net 4: 327680.. 328703: 491520.. 492543: 1024: 0001: net 5: 0.. 65535: 2883584.. 2949119: 65536: 0000: net 6: 262144.. 327679: 2949120.. 3014655: 65536: 0000: last,net test_simple: 5 extents found [root@client client]# du test_simple 263176 test_simple [root@client client]# strace -o report lfs migrate -c2 test_simple [root@client client]# filefrag -v test_simple Filesystem type is: bd00bd0 File size of test_simple is 672137216 (656384 blocks of 1024 bytes) ext: device_logical: physical_offset: length: dev: flags: 0: 0.. 3: 433676.. 433679: 4: 0001: net 1: 512.. 1023: 457244.. 457755: 512: 0001: net 2: 1024.. 1535: 461812.. 462323: 512: 0001: net 3: 1536.. 2047: 457756.. 458267: 512: 0001: net 4: 2048.. 3583: 462848.. 464383: 1536: 0001: net 5: 3584.. 4095: 397824.. 398335: 512: 0001: net 6: 4096.. 24575: 499712.. 520191: 20480: 0001: net 7: 24576.. 66559: 2629632.. 2671615: 41984: 0001: net 8: 262656.. 328703: 2679296.. 2745343: 66048: 0001: net 9: 0.. 1023: 473088.. 474111: 1024: 0000: net 10: 1024.. 1535: 466432.. 466943: 512: 0000: net 11: 1536.. 2047: 475136.. 475647: 512: 0000: net 12: 2048.. 3071: 474112.. 475135: 1024: 0000: net 13: 3072.. 3583: 475648.. 476159: 512: 0000: net 14: 3584.. 66047: 3014656.. 3077119: 62464: 0000: net 15: 262144.. 327679: 3088384.. 3153919: 65536: 0000: last,net test_simple: 16 extents found [root@client client]# du test_simple 263692 test_simple </pre> </div></div> <p>block_diff = 263692 - 263176 = <b>516kb</b></p> <p><b>"lfs migrate" strace:</b></p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lseek(3, 4096, SEEK_DATA) = 524288 lseek(3, 524288, SEEK_HOLE) = 135266304 pread64(3, "]\220\311\337\376 T@\3622\314\231\3\0100\202]\31j\246u\177\256\255\307\250\0\333\311\301\346\251"..., 1048576, 524288) = 1048576 pwrite64(4, "]\220\311\337\376 T@\3622\314\231\3\0100\202]\31j\246u\177\256\255\307\250\0\333\311\301\346\251"..., 1048576, 524288) = 1048576 ... pwrite64(4, "\266\270\20\357\243\366#jC\217\333\325\260V\314;\274b\204\250\4s\\\231\6\222\34\300\30n]\27"..., 1048576, 133693440) = 1048576 pread64(3, "\325/\377m\316\251P\2723\27*%1\205\216\365; \tv$\241X\31&gt;\342\331f\265td\353"..., 1048576, 134742016) = 1048576 pwrite64(4, "\325/\377m\316\251"..., 1048576, 134742016) = 1048576 &lt;------ 134742016 + 1048576 = 135790592 != seek_hole (135266304) lseek(3, 135790592, SEEK_DATA) = 537395200 lseek(3, 537395200, SEEK_HOLE) = 672137216 </pre> </div></div> LU-16948

sparse file migration broken

Bug Minor Open Unresolved Etienne Aujames Etienne Aujames Thu, 6 Jul 2023 15:42:24 +0000 Wed, 2 Aug 2023 18:47:42 +0000 Lustre 2.16.0 Lustre 2.15.3 0 5 <p>"On 2.15.3 sparse file migration with a stripe count &gt; 2 does not keep the same number of blocks if the extents are non-aligned on the stripe_size (on master the extent must be &gt; 64M):"</p> <p>Etienne, I don't follow the second part here - What is the bug on master exactly?</p> <p>Also, did you mean stripe count &gt; 1 ?</p> <p>Sorry, my mistake, the stripe_count is not the culprit (I can reproduce this with stripe_count=1).</p> <p>"Etienne AUJAMES &lt;eaujames@ddn.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/51593" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/51593</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-16948" title="sparse file migration broken" class="issue-link" data-issue-key="LU-16948">LU-16948</a> utils: fix lfs migrate for sparse files<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5f0826104c694db56ec22d40b094f8a74e9d9ffe</p> <p>This bug is linked to the buffer_size, on master the buffer_size is &gt; 64M: <a href="https://jira.whamcloud.com/browse/LU-16587" title="Make lfs migrate faster" class="issue-link" data-issue-key="LU-16587"><del>LU-16587</del></a> utils: give lfs migrate a larger buffer (<a href="https://review.whamcloud.com/c/fs/lustre-release/+/50118" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/50118</a>)</p> <p>On master the block_diff should be something like this:<br/> block_diff = buffer_size - (segment_size % buffer_size) = 64M - 512K</p> Related LU-16587 Development Rank 1|i03ptr: Rank (Obsolete) 9223372036854775807 Severity