Details
-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
Lustre 2.15.4
-
None
-
3
-
9223372036854775807
Description
lfs_migrate -A seems to be broken if the file to be migrated uses a PFL layout: option -A is basically ignored and lfs migrate --copy is used, the file is migrated using the same PFL layout, and if the previous OST(s) in the old layout are disabled (eg. max_create_count=0), always the next available OST is picked up, leading to poor space balancing.
Example:
# lfs getstripe /oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-0.05_n1.0_thermal_perlmt_2/nflux1/n1.0/beta4.5_U6_mu-0.045/2024_01_04_113.h5
/oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-0.05_n1.0_thermal_perlmt_2/nflux1/n1.0/beta4.5_U6_mu-0.045/2024_01_04_113.h5
lcm_layout_gen: 2
lcm_mirror_count: 1
lcm_entry_count: 2
lcme_id: 1
lcme_mirror_id: 0
lcme_flags: init
lcme_extent.e_start: 0
lcme_extent.e_end: 2199023255552
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_pattern: raid0
lmm_layout_gen: 0
lmm_stripe_offset: 69
lmm_objects:
- 0: { l_ost_idx: 69, l_fid: [0xc00000400:0x3dd08a:0x0] }
lcme_id: 2
lcme_mirror_id: 0
lcme_flags: 0
lcme_extent.e_start: 2199023255552
lcme_extent.e_end: EOF
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: raid0
lmm_layout_gen: 0
lmm_stripe_offset: -1
# echo /oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-0.05_n1.0_thermal_perlmt_2/nflux1/n1.0/beta4.5_U6_mu-0.045/2024_01_04_113.h5 | lfs_migrate -A -C 20 -D --non-block --no-rsync -y -v
/oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-0.05_n1.0_thermal_perlmt_2/nflux1/n1.0/beta4.5_U6_mu-0.045/2024_01_04_113.h5: stripe_count=1,stripe_size=1048576
lfs migrate -D --non-block --copy /oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-0.05_n1.0_thermal_perlmt_2/nflux1/n1.0/beta4.5_U6_mu-0.045/2024_01_04_113.h5 "/oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-
done
# lfs getstripe /oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-0.05_n1.0_thermal_perlmt_2/nflux1/n1.0/beta4.5_U6_mu-0.045/2024_01_04_113.h5
/oak/stanford/orgs/simes/jxding/jxding-desktop-Data-DQMC/8x8_tp-0.05_n1.0_thermal_perlmt_2/nflux1/n1.0/beta4.5_U6_mu-0.045/2024_01_04_113.h5
lcm_layout_gen: 3
lcm_mirror_count: 1
lcm_entry_count: 2
lcme_id: 1
lcme_mirror_id: 0
lcme_flags: init
lcme_extent.e_start: 0
lcme_extent.e_end: 2199023255552
lmm_stripe_count: 1
lmm_stripe_size: 1048576
lmm_pattern: raid0
lmm_layout_gen: 0
lmm_stripe_offset: 193
lmm_objects:
- 0: { l_ost_idx: 193, l_fid: [0x3340000402:0x72d3ab:0x0] }
lcme_id: 2
lcme_mirror_id: 0
lcme_flags: 0
lcme_extent.e_start: 2199023255552
lcme_extent.e_end: EOF
lmm_stripe_count: 8
lmm_stripe_size: 1048576
lmm_pattern: raid0
lmm_layout_gen: 0
lmm_stripe_offset: -1
193 is the next available OST index with max_create_count=2000. All migrated files are migrated to this OST which will fill up dramatically when doing a lot of migrations.
What we want is the previous behavior: lfs_migrate -A should restripe the file automatically to a plain layout as documented with OST space balancing controlled by the MDT. Was this change in 2.15 intended? I've seen a few tickets to support PFL, but while PFL is awesome, lfs_migrate -A is still be useful to migrate old PFL files to plain layout knowing the file size in advance and using stripe count = sqrt(filesize_in_GB) + 1 as documented in the man page.
In lfs_migrate, changing:
450 if ! $OPT_COMP && [ $\{layout_info[$l_comp_count]} -gt 0 ]; then
451 layout+="--copy $OLDNAME"
452 OPT_COMP=true
453 fi
to:
450 if ! $OPT_COMP && ! $OPT_AUTOSTRIPE && [ $\{layout_info[$l_comp_count]} -gt 0 ]; then
451 layout+="--copy $OLDNAME"
452 OPT_COMP=true
453 fi
seems to restore 2.12's behavior.
It's just an issue with the lfs_migrate script I believe, so no big deal, and let me know if I missed something obvious here.