[LU-14459] DNE3: directory auto split during create

• Type: Improvement
• Resolution: Unresolved
• Priority: Major
• Labels: dne3
• Assignee: Lai Siyao
• Reporter: Andreas Dilger

Description

Directory auto-split should be done when the dir_split_count is hit during file creation to minimize the number of entries that need to be moved, rather than afterward. Otherwise, the entries are being created on a single MDT first, and only being migrated on a later directory access. This is sub-optimal for two reasons:

• the initial file creation will be limited to a single MDT and not distribute the load across multiple MDTs, so just entry migration will still leave all the inodes on the original MDT
• the post-creation migration will need to move many more files/entries to the other MDTs than if the files are created on the correct MDT to start with

Issue Links

• duplicates: LU-14467 Allow split metadata across MDTs during file creation
• is related to: LU-14464 Auto restripe triggers with none of the reasons
• is related to: LU-14466 metadata performance slows if the metadata migration is process is running
• is related to: LU-15692 performance regressions for files in stripe directory
• is related to: LU-15720 imbalanced file creation in 'crush' striped directory
• is related to: LU-15502 mkdir returns -EBADF if default LMV is set in 2.12 client <-> 2.14 MDS system
• is related to: LU-14146 Massive directory metadata operation performance decrease
• is related to: LU-11025 DNE3: directory restripe

Comments

[Andreas Dilger]

The performance impact of the auto-split during mdtest is too high, since this moves all of the entries while the benchmark is running, and DNE distributed transactions cause a lot of sync operations. It would be better to delay the entry move, like put migrations for directory split onto a queue, and only move those entries when the RPC threads are idle (maybe with a "low priority" RPC queue that is only handled when high/normal RPCs are finished?).

That would avoid blocking the benchmark RPCs, but still split the directory early so that the metadata IOPS of the directory increase during the test, and "most" entries are created on the right MDT, with fewer entries need to be moved later on.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43291/
Subject: LU-14459 mdt: support fixed directory layout
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4c2514f4832801374092f3a48c755248af345566

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43290/
Subject: LU-14459 mdt: restripe parent may be a stripe
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a84efc8607ae8057499a8800699f336e821b03d8

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43289/
Subject: LU-14459 llite: reset pfid after dir migration
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: abbe545a63b304e803ee62443dd65f1feeed15cd

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/43684/
Subject: LU-14459 lmv: change default hash type to crush
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: bb60caa1c6e7c14c201916dc0423442d10c86a27

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43684
Subject: LU-14459 lmv: change default hash type to crush
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e859f57768b8b7ee2e44924c2b36e39a7f0f323a

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43291
Subject: LU-14459 mdt: support fixed directory layout
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0c0995c016c49f21b1dea938cdb71c6ccb7bf658

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43290
Subject: LU-14459 mdt: restripe parent may be a stripe
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 93711b9e00d28d235c2b10409bf135eb6c6e5944

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/43289
Subject: LU-14459 llite: init stripe pfid after dir split
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 768bc1b9e93ebc5c3c53612287abd3ca136c622c

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/42064
Subject: LU-14459 mdt: trigger dir split in create/open
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0edbcc3ce6de3f2c625c5f3be6805565a81fab51