[LU-16500] "lfs migrate <file>" preserves specific layout too much

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Affects Version/s: Lustre 2.16.0
• Severity: 3
• Assignee: Jian Yu
• Reporter: Andreas Dilger

Description

Running "lfs migrate <file>" without any SETSTRIPE arguments to balance space usage keeps the PFL file layout, but preserves the OST selection "too well". The "lfs migrate" command is intended to be used for space balancing OSTs, so it would be expected to change the starting OST and other OSTs in use to better balance, but the OSTs are preserved exactly and this makes the migration virtually useless in this case:

# lfs df
UUID                   1K-blocks        Used   Available Use% Mounted on
testfs-MDT0000_UUID       125056        2268      111552   2% /mnt/testfs[MDT:0]
testfs-OST0000_UUID       313104        1664      284280   1% /mnt/testfs[OST:0]
testfs-OST0001_UUID       313104      206464       79480  73% /mnt/testfs[OST:1]
testfs-OST0002_UUID       313104        1668      284276   1% /mnt/testfs[OST:2]
testfs-OST0003_UUID       313104        1668      284276   1% /mnt/testfs[OST:3]

filesystem_summary:      1252416      211464      932312  19% /mnt/testfs

# lfs setstripe -E 1M -c 1 -E 16M -c 1 -E eof -c 1 /mnt/testfs/junk3
# fallocate -l 17M /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5d:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc1:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x22:0x0] }
# lfs migrate /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5e:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc2:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x23:0x0] }
# lfs migrate /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5f:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc3:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x24:0x0] }
# lfs migrate /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x60:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc4:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x25:0x0] }

One would expect with the OST0001 imbalance that it would be skipped for allocation, but the "lfs migrate" command must be using a specific layout. Instead, it should be clearing the specific OST indices from the layout and only using it as a template.

Creating a new file correctly avoids the full OST, so this isn't a problem with the QOS space balancing:

# lfs setstripe -E 1M -c 1 -E 16M -c 1 -E eof -c 1 /mnt/testfs/junk3.2
# fallocate -l 17M /mnt/testfs/junk3.2
# lfs getstripe /mnt/testfs/junk3.2 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x62:0x0] }
      - 0: { l_ost_idx: 3, l_fid: [0x100030000:0x5f:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x27:0x0] }

Issue Links

• is duplicated by: LU-17697 lfs_migrate -A broken in 2.15 if source file uses PFL
• is related to: LU-16522 "lfs setstripe -i N" with deactivated OST(s) always picks next active OST
• is related to: LU-15834 FLR: "lfs mirror extend" should take current OSTs into account
• is related to: LU-13062 OST offset defaults to 0 when coying a PFL via xattrs

Comments

[Peter Jones]

Merged for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54600/
Subject: LU-16500 utils: 'lfs migrate' should select new OSTs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 2007ab4709acaef0397df15c9f4cf4387844ba9c

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/54600
Subject: LU-16500 utils: 'lfs migrate' should select new OSTs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5bba268af1ba040aa29f666baf8c958028c2e1a8

[Andreas Dilger]

The landed patch didn't totally fix the problem. For files with multiple components, running "lfs migrate FILE" repeatedly on the same file only changes the last component of the file, because llapi_layout_ost_index_set() is only changing cur_comp.

I will push a patch to reset the OST index for all components of the file.