Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16500

"lfs migrate <file>" preserves specific layout too much

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • Lustre 2.16.0
    • Lustre 2.16.0
    • None
    • 3
    • 9223372036854775807

    Description

      Running "lfs migrate <file>" without any SETSTRIPE arguments to balance space usage keeps the PFL file layout, but preserves the OST selection "too well". The "lfs migrate" command is intended to be used for space balancing OSTs, so it would be expected to change the starting OST and other OSTs in use to better balance, but the OSTs are preserved exactly and this makes the migration virtually useless in this case:

      # lfs df
      UUID                   1K-blocks        Used   Available Use% Mounted on
      testfs-MDT0000_UUID       125056        2268      111552   2% /mnt/testfs[MDT:0]
      testfs-OST0000_UUID       313104        1664      284280   1% /mnt/testfs[OST:0]
      testfs-OST0001_UUID       313104      206464       79480  73% /mnt/testfs[OST:1]
      testfs-OST0002_UUID       313104        1668      284276   1% /mnt/testfs[OST:2]
      testfs-OST0003_UUID       313104        1668      284276   1% /mnt/testfs[OST:3]
      
      filesystem_summary:      1252416      211464      932312  19% /mnt/testfs
      
      # lfs setstripe -E 1M -c 1 -E 16M -c 1 -E eof -c 1 /mnt/testfs/junk3
      # fallocate -l 17M /mnt/testfs/junk3
      # lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
            - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5d:0x0] }
            - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc1:0x0] }
            - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x22:0x0] }
      # lfs migrate /mnt/testfs/junk3
      # lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
            - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5e:0x0] }
            - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc2:0x0] }
            - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x23:0x0] }
      # lfs migrate /mnt/testfs/junk3
      # lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
            - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x5f:0x0] }
            - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc3:0x0] }
            - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x24:0x0] }
      # lfs migrate /mnt/testfs/junk3
      # lfs getstripe /mnt/testfs/junk3 | grep l_ost_idx
            - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x60:0x0] }
            - 0: { l_ost_idx: 1, l_fid: [0x100010000:0xc4:0x0] }
            - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x25:0x0] }
      

      One would expect with the OST0001 imbalance that it would be skipped for allocation, but the "lfs migrate" command must be using a specific layout. Instead, it should be clearing the specific OST indices from the layout and only using it as a template.

      Creating a new file correctly avoids the full OST, so this isn't a problem with the QOS space balancing:

      # lfs setstripe -E 1M -c 1 -E 16M -c 1 -E eof -c 1 /mnt/testfs/junk3.2
      # fallocate -l 17M /mnt/testfs/junk3.2
      # lfs getstripe /mnt/testfs/junk3.2 | grep l_ost_idx
            - 0: { l_ost_idx: 0, l_fid: [0x100000000:0x62:0x0] }
            - 0: { l_ost_idx: 3, l_fid: [0x100030000:0x5f:0x0] }
            - 0: { l_ost_idx: 2, l_fid: [0x100020000:0x27:0x0] }
      

      Attachments

        Issue Links

          Activity

            People

              yujian Jian Yu
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: