[LU-9007] Improved object allocator for FLR composite files

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.12.0
• Labels: FLR2
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: Joseph Gmitter (Inactive)

Description

The current MDS object allocator is designed only to allocate objects for one file at the time the file is first created. For progressive file layouts, at a minimum the allocator will need to be enhanced in order to avoid allocating objects on OSTs that are already part of a file's other components. If files have multiple objects allocated to the same OSTs before objects are allocated from unused OSTs, there may be a significant performance loss due to oversubscribing the bandwidth on that OST compared to the other OSTs. The only exception may be for a fully-striped component at the end of the file (see Example Progressive Layouts for more detail), where it would be acceptable to allocate objects across all of the available OSTs to maximize the bandwidth available for the file.

Issue Links

• is related to: LU-15834 "lfs mirror extend" should take current OSTs into account
• is related to: LU-15841 sanity-flr test 47 is failing with 'component 65537, 131075, 196613 have objects allocated on duplicated OSTs'
• is related to: LU-11238 sanity-flr test 47 fails with “component 131075 objects allocated on 0, shouldn't on OST0”
• is related to: LU-8998 Progressive File Layout (PFL)
• is related to: LU-10158 FLR: Define a replica choosing policy function