[LU-9007] Improved object allocator for FLR composite files

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.12.0
• Labels: FLR2
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: Joseph Gmitter (Inactive)

Description

The current MDS object allocator is designed only to allocate objects for one file at the time the file is first created. For progressive file layouts, at a minimum the allocator will need to be enhanced in order to avoid allocating objects on OSTs that are already part of a file's other components. If files have multiple objects allocated to the same OSTs before objects are allocated from unused OSTs, there may be a significant performance loss due to oversubscribing the bandwidth on that OST compared to the other OSTs. The only exception may be for a fully-striped component at the end of the file (see Example Progressive Layouts for more detail), where it would be acceptable to allocate objects across all of the available OSTs to maximize the bandwidth available for the file.

Issue Links

• is related to: LU-15834 FLR: "lfs mirror extend" should take current OSTs into account
• is related to: LU-15841 sanity-flr test 47 is failing with 'component 65537, 131075, 196613 have objects allocated on duplicated OSTs'
• is related to: LU-11238 sanity-flr test 47 fails with “component 131075 objects allocated on 0, shouldn't on OST0”
• is related to: LU-8998 Progressive File Layout (PFL)
• is related to: LU-10158 FLR2: improve mirror selection policy functions

Comments

[Peter Jones]

Landed for 2.12

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/32813/
Subject: LU-9007 lod: get rid of comp ost in use array
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a277952c65d4aad1abb9ac9f759af16a43902068

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/32404/
Subject: LU-9007 lod: improve obj alloc for FLR file
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: fabf3fe7ac06d916d8c433a99f1f4a4bd3632638

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32813
Subject: LU-9007 lod: get rid of comp ost in use array
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 73c65d7e3402e2465a0b1042eb7ccaf185730b87

[Gerrit Updater]

Bobi Jam ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32404
Subject: LU-9007 lod: improve obj alloc for FLR file
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 1a0c093c55d034ba6013f05f7c2a68664d3d0901