[LU-9846] Overstriping - more than stripe per OST per component

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.13.0
• Labels: IO500
• Severity: 3
• Assignee: Patrick Farrell (Inactive)
• Reporter: Patrick Farrell (Inactive)

Description

'Overstriping' is my term for allowing more than one stripe of a particular file component to be placed on a particular OST.

Justification:
Lock ahead is designed to address the case were we are limited to a single shared file, but each OST is significantly faster than one client. In a shared file situation, LDLM locking behavior limits us to writing with one client to each OST, so we are unable to fully drive each OST for the shared file. This is an interesting enough case for Cray to drive all the work in lock ahead use a library to achieve it.

If we can put multiple stripes of the file on a single OST, we can essentially achieve the same thing, with far less effort. For a variety of reasons, this doesn't remove the need for lockahead (Primarily because we cannot necessarily redefine file striping when we want to write to it), but it is much simpler, and highly desirable for that reason. In addition to the MPIIO aggregation case where we have well controlled I/O and are trying to maximize OST utilization, adding more stripes to a shared file also helps in cases where I/O is poorly controlled, so there are effectively more locks for the badly behaved writers to contend for.

So, in short, I think it would be very, very desirable if, in a controlled manner, we could ask for more than one stripe to be on a given OST. A simple example is something like "8 stripes but only on these 2 OSTs", giving 4 stripes per OST (and allowing 4 client writers per OST with no fancy locking work).

A note about PFL:
While pfl lets you create separate components using the same OST, that's not a viable solution for the case I'm looking for, which is just > 1 stripe on a given OST. The idea being to write to all stripes in parallel, but wanting > 1 writer to be pointed at each OST. PFL would require a neverending series of components to do that.

Patch to implement this follows.

Issue Links

• is related to: LU-1658 Review consistantly fails when running with 300 osts - wide stripe testing.
• is related to: LU-11784 PFL layouts can exceed EA size limits
• is related to: LU-11868 ZFS ea size limited to 32K
• is related to: LU-12273 DNE3: Metadata overstriping
• is related to: LU-11690 LBUG with very wide striping: lod_ea_store_resize()) ASSERTION( round <= lov_mds_md_size(2000, (0x0BD30000 | 0x0BD0))
• is related to: LU-10070 PFL self-extending file layout

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35089/
Subject: LU-9846 test: a test number fix
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 360cb33fccd2fc7a0dc392afbb780ac1284b403a

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35095/
Subject: LU-9846 utils: hash may be overridden in 'lfs setdirstripe'
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 44405d4d6f7c0a77d540d6a114050ccabd0a4e9f

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/35095
Subject: LU-9846 utils: hash may be overridden in 'lfs setdirstripe'
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 44a3f74059235aac9428311fb8cb1c4e44a6babf

[Gerrit Updater]

Vitaly Fertman ([e-mail]) uploaded a new patch: https://review.whamcloud.com/35089
Subject: LU-9846 test: a test number fix
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 50e4b01c8c0b0bfa4a20b5c421f04dd974274527

[Peter Jones]

Landed for 2.13

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/28425/
Subject: LU-9846 lod: Add overstriping support
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 591a9b4cebc510ff51f0fdb944e5a81f08fdaf62

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/34743/
Subject: LU-9846 obd: Add overstriping CONNECT flag
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5d085745af43bd15e6b7ea728491600411833b2a

[Gerrit Updater]

Patrick Farrell ([e-mail]) uploaded a new patch: https://review.whamcloud.com/34743
Subject: LU-9846 obd: Add overstriping CONNECT flag
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 1b17db2eee420e32944e8668c85c92d766fd6d82

[Gerrit Updater]

Patrick Farrell ([e-mail]) uploaded a new patch: https://review.whamcloud.com/34189
Subject: LU-9846 tests: Overstriping test
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 637dce43039fc94c5b1695a395f02f61f28162c9

[Patrick Farrell (Inactive)]

Decided to split out raising the stripe count limit from the feature patch, as they're different and may cause different issues.

Both still depend on landing https://jira.whamcloud.com/browse/LU-11690 first.