Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14459

DNE3: directory auto split during create

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • None
    • 9223372036854775807

    Description

      Directory auto-split should be done when the dir_split_count is hit during file creation to minimize the number of entries that need to be moved, rather than afterward. Otherwise, the entries are being created on a single MDT first, and only being migrated on a later directory access. This is sub-optimal for two reasons:

      • the initial file creation will be limited to a single MDT and not distribute the load across multiple MDTs, so just entry migration will still leave all the inodes on the original MDT
      • the post-creation migration will need to move many more files/entries to the other MDTs than if the files are created on the correct MDT to start with

      Attachments

        Issue Links

          Activity

            [LU-14459] DNE3: directory auto split during create

            The performance impact of the auto-split during mdtest is too high, since this moves all of the entries while the benchmark is running, and DNE distributed transactions cause a lot of sync operations. It would be better to delay the entry move, like put migrations for directory split onto a queue, and only move those entries when the RPC threads are idle (maybe with a "low priority" RPC queue that is only handled when high/normal RPCs are finished?).

            That would avoid blocking the benchmark RPCs, but still split the directory early so that the metadata IOPS of the directory increase during the test, and "most" entries are created on the right MDT, with fewer entries need to be moved later on.

            adilger Andreas Dilger added a comment - The performance impact of the auto-split during mdtest is too high, since this moves all of the entries while the benchmark is running, and DNE distributed transactions cause a lot of sync operations. It would be better to delay the entry move, like put migrations for directory split onto a queue, and only move those entries when the RPC threads are idle (maybe with a "low priority" RPC queue that is only handled when high/normal RPCs are finished?). That would avoid blocking the benchmark RPCs, but still split the directory early so that the metadata IOPS of the directory increase during the test, and "most" entries are created on the right MDT, with fewer entries need to be moved later on.

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43291/
            Subject: LU-14459 mdt: support fixed directory layout
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 4c2514f4832801374092f3a48c755248af345566

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43291/ Subject: LU-14459 mdt: support fixed directory layout Project: fs/lustre-release Branch: master Current Patch Set: Commit: 4c2514f4832801374092f3a48c755248af345566

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43290/
            Subject: LU-14459 mdt: restripe parent may be a stripe
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: a84efc8607ae8057499a8800699f336e821b03d8

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43290/ Subject: LU-14459 mdt: restripe parent may be a stripe Project: fs/lustre-release Branch: master Current Patch Set: Commit: a84efc8607ae8057499a8800699f336e821b03d8

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43289/
            Subject: LU-14459 llite: reset pfid after dir migration
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: abbe545a63b304e803ee62443dd65f1feeed15cd

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43289/ Subject: LU-14459 llite: reset pfid after dir migration Project: fs/lustre-release Branch: master Current Patch Set: Commit: abbe545a63b304e803ee62443dd65f1feeed15cd

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43684/
            Subject: LU-14459 lmv: change default hash type to crush
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: bb60caa1c6e7c14c201916dc0423442d10c86a27

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/43684/ Subject: LU-14459 lmv: change default hash type to crush Project: fs/lustre-release Branch: master Current Patch Set: Commit: bb60caa1c6e7c14c201916dc0423442d10c86a27

            Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43684
            Subject: LU-14459 lmv: change default hash type to crush
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: e859f57768b8b7ee2e44924c2b36e39a7f0f323a

            gerrit Gerrit Updater added a comment - Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43684 Subject: LU-14459 lmv: change default hash type to crush Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: e859f57768b8b7ee2e44924c2b36e39a7f0f323a

            Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43291
            Subject: LU-14459 mdt: support fixed directory layout
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 0c0995c016c49f21b1dea938cdb71c6ccb7bf658

            gerrit Gerrit Updater added a comment - Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43291 Subject: LU-14459 mdt: support fixed directory layout Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 0c0995c016c49f21b1dea938cdb71c6ccb7bf658

            Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43290
            Subject: LU-14459 mdt: restripe parent may be a stripe
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 93711b9e00d28d235c2b10409bf135eb6c6e5944

            gerrit Gerrit Updater added a comment - Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43290 Subject: LU-14459 mdt: restripe parent may be a stripe Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 93711b9e00d28d235c2b10409bf135eb6c6e5944

            Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43289
            Subject: LU-14459 llite: init stripe pfid after dir split
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 768bc1b9e93ebc5c3c53612287abd3ca136c622c

            gerrit Gerrit Updater added a comment - Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/43289 Subject: LU-14459 llite: init stripe pfid after dir split Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 768bc1b9e93ebc5c3c53612287abd3ca136c622c

            Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/42064
            Subject: LU-14459 mdt: trigger dir split in create/open
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 0edbcc3ce6de3f2c625c5f3be6805565a81fab51

            gerrit Gerrit Updater added a comment - Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/42064 Subject: LU-14459 mdt: trigger dir split in create/open Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 0edbcc3ce6de3f2c625c5f3be6805565a81fab51

            People

              laisiyao Lai Siyao
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: