[LU-16587] Make lfs migrate faster

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Assignee: Nathan Rutman
• Reporter: Nathan Rutman

Description

Most of the time lfs migrate uses a 1MB buffer (stripe size) to copy data. This is terribly slow.

In my testing I see improved performance up to 64M buffers.

[root@kjlmo4n00 16G]# lfs getstripe 16G.1
16G.1
lmm_stripe_count:  1
lmm_stripe_size:   1048576
lmm_pattern:       raid0
lmm_layout_gen:    11
lmm_stripe_offset: 1
lmm_pool:          flash
	obdidx		 objid		 objid		 group
	     1	        394422	      0x604b6	             0

[root@kjlmo4n00 16G]# time lfs migrate -S 1M -p flash 16G.1

real	0m25.341s
user	0m0.001s
sys	0m2.606s
[root@kjlmo4n00 16G]# time /root/tools/lfs_nzr migrate -S 1M -p flash 16G.1

real	0m6.526s
user	0m0.000s
sys	0m2.177s

You can force lfs to use a bigger buffer by increasing stripe size and you can see similar improvements.

Issue Links

• is related to: LU-16948 sparse file migration broken
• is related to: LU-16789 "lfs migrate" to use AIO/DIO or io_uring (kernel 5.1+)