[LU-6148] Strided lock proposal - Feature proposal for 2.8

• Status: Resolved
• Project: Lustre
• Type: New Feature
• Priority: Minor
• Reporter: Patrick Farrell (Inactive)
• Assignee: WC Triage
• Resolution: Won't Do
• Severity: 3

Description

This ticket is a placeholder for further information forthcoming.

Briefly, strided locking is a proposed new feature to make improved shared file IO performance possible in certain situations. This is being proposed at the Lustre developers meeting at LLNL (currently ongoing).

Details will be forthcoming, for now, this ticket is mostly to allow me to push the prototype to Gerrit for ease of viewing.

Issue Links

• is related to: LU-6179 Lock ahead - Request extent locks fro...
• is related to: LU-6180 Fix page cache flushing when group lo...

Comments

[Gerrit Updater]

Patrick Farrell ([e-mail]) uploaded a new patch: http://review.whamcloud.com/13498
Subject: LU-6148 ldlm: Strided locking prototype
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 590c2b30d66021aac4f4a31fb05754036092c0dd

[Patrick Farrell (Inactive)]

Prototype/proof of concept here:
http://review.whamcloud.com/13498

[Patrick Farrell (Inactive)]

Presentation slides from developer's conference.

[Alex Zhuravlev]

what about group locks?

[Patrick Farrell (Inactive)]

Alex - The presentation PDF explains why not in a bit more detail, giving a few reasons.

The main reason is what I'm calling write visibility:

Client A and B both have the same group lock.
Client A reads some pages, then client B writes to those pages.
Client A reads the pages again, but because it has a lock on the file, it does not realize the pages have been modified & will re-use its cached pages.

Some of the more complex data formats used in HPC - Such as HFD5 - write out in file metadata as part of writing out a file, and it requires write visibility in the course of the IO.

[Patrick Farrell (Inactive)]

Jinshan suggested implementing an ioctl for user space to request LDLM locks, allowing user space to 'lock ahead' of the actual IO, and we agreed I'd implement that and test it for performance before considering strided locks further. If that can get most of the performance of file per process, then there isn't much point to strided locks, and we do not need to add the complexity. We agreed there that if that doesn't close the gap, strided locks are worth considering.

A prototype of lock ahead is here:
LU-6179

This ticket's going to be on hold until we have more results there.

[Andreas Dilger]

This approach was abandoned in favour of lockahead.