[LU-9846] Overstriping - more than stripe per OST per component

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.13.0
• Labels: IO500
• Severity: 3
• Assignee: Patrick Farrell (Inactive)
• Reporter: Patrick Farrell

Description

'Overstriping' is my term for allowing more than one stripe of a particular file component to be placed on a particular OST.

Justification:
Lock ahead is designed to address the case were we are limited to a single shared file, but each OST is significantly faster than one client. In a shared file situation, LDLM locking behavior limits us to writing with one client to each OST, so we are unable to fully drive each OST for the shared file. This is an interesting enough case for Cray to drive all the work in lock ahead use a library to achieve it.

If we can put multiple stripes of the file on a single OST, we can essentially achieve the same thing, with far less effort. For a variety of reasons, this doesn't remove the need for lockahead (Primarily because we cannot necessarily redefine file striping when we want to write to it), but it is much simpler, and highly desirable for that reason. In addition to the MPIIO aggregation case where we have well controlled I/O and are trying to maximize OST utilization, adding more stripes to a shared file also helps in cases where I/O is poorly controlled, so there are effectively more locks for the badly behaved writers to contend for.

So, in short, I think it would be very, very desirable if, in a controlled manner, we could ask for more than one stripe to be on a given OST. A simple example is something like "8 stripes but only on these 2 OSTs", giving 4 stripes per OST (and allowing 4 client writers per OST with no fancy locking work).

A note about PFL:
While pfl lets you create separate components using the same OST, that's not a viable solution for the case I'm looking for, which is just > 1 stripe on a given OST. The idea being to write to all stripes in parallel, but wanting > 1 writer to be pointed at each OST. PFL would require a neverending series of components to do that.

Patch to implement this follows.

Issue Links

• is related to: LU-1658 Review consistantly fails when running with 300 osts - wide stripe testing.
• is related to: LU-11784 PFL layouts can exceed EA size limits
• is related to: LU-11868 ZFS ea size limited to 32K
• is related to: LU-12273 DNE3: Metadata overstriping
• is related to: LU-11690 LBUG with very wide striping: lod_ea_store_resize()) ASSERTION( round <= lov_mds_md_size(2000, (0x0BD30000 | 0x0BD0))
• is related to: LU-10070 PFL self-extending file layout