Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16500

"lfs migrate <file>" preserves specific layout too much

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • Lustre 2.16.0
    • Lustre 2.16.0
    • None
    • 3
    • 9223372036854775807

      Running "lfs migrate <file>" without any SETSTRIPE arguments to balance space usage keeps the PFL file layout, but preserves the OST selection "too well". The "lfs migrate" command is intended to be used for space balancing OSTs, so it would be expected to change the starting OST and other OSTs in use to better balance, but the OSTs are preserved exactly and this makes the migration virtually useless in this case:

      # lfs df
UUID                   1K-blocks        Used   Available Use% Mounted on
testfs-MDT0000_UUID       125056        2268      111552   2% /mnt/testfs[MDT:0]
testfs-OST0000_UUID       313104        1664      284280   1% /mnt/testfs[OST:0]
testfs-OST0001_UUID       313104      206464       79480  73% /mnt/testfs[OST:1]
testfs-OST0002_UUID       313104        1668      284276   1% /mnt/testfs[OST:2]
testfs-OST0003_UUID       313104        1668      284276   1% /mnt/testfs[OST:3]

filesystem_summary:      1252416      211464      932312  19% /mnt/testfs

# lfs setstripe -E 1M -c 1 -E 16M -c 1 -E eof -c 1 /mnt/testfs/junk3
# fallocate -l 17M /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5d:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc1:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x22:0x0] }
# lfs migrate /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5e:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc2:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x23:0x0] }
# lfs migrate /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5f:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc3:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x24:0x0] }
# lfs migrate /mnt/testfs/junk3
# lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x60:0x0] }
      - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc4:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x25:0x0] }

      One would expect with the OST0001 imbalance that it would be skipped for allocation, but the "lfs migrate" command must be using a specific layout. Instead, it should be clearing the specific OST indices from the layout and only using it as a template.

      Creating a new file correctly avoids the full OST, so this isn't a problem with the QOS space balancing:

      # lfs setstripe -E 1M -c 1 -E 16M -c 1 -E eof -c 1 /mnt/testfs/junk3.2
# fallocate -l 17M /mnt/testfs/junk3.2
# lfs getstripe /mnt/testfs/junk3.2 | grep l_ost_idx
      - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x62:0x0] }
      - 0: { l_ost_idx: 3, l_fid: [0x100030000:0x5f:0x0] }
      - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x27:0x0] }

            yujian Jian Yu
            adilger Andreas Dilger
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: