[LU-6179] Lock ahead - Request extent locks from userspace

• Type: New Feature
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.11.0
• Labels: bgti patch
• Assignee: Patrick Farrell (Inactive)
• Reporter: Patrick Farrell (Inactive)

Description

At the recent developers conference, Jinshan proposed a different method of approaching the performance problems described in LU-6148.

Instead of introducing a new type of LDLM lock matching, we'd like to make it possible for user space to explicitly request LDLM locks asynchronously from the IO.

I've implemented a prototype version of the feature and will be uploading it for comments. I'll explain the state of the current version in a comment momentarily.

Issue Links

• is blocking: LU-9962 Possible file size glimpse issue
• is related to: LU-10136 sanity test_255c: Ladvise test11 failed, 255
• is related to: LUDOC-379 Document ladvise lockahead feature
• is related to: LU-6181 Fix lock contention discovery to disable extend lock growth
• is related to: LU-11618 implement ladvise rpc_size for optimized performance
• is related to: LU-6148 Strided lock proposal - Feature proposal for 2.8
• is related to: LU-7225 change ladvise wire protocol for lockahead and future usage

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/38179/
Subject: LU-6179 llite: Remove last lockahead old compat
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 12a0c7b5944d9e48e38416c7cac2cde153e3148b

[Gerrit Updater]

Patrick Farrell ([e-mail]) uploaded a new patch: https://review.whamcloud.com/38179
Subject: LU-6179 llite: Remove last lockahead old compat
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d23ed5713d41b5d8c69a989bb2600d83bf701c31

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/38109/
Subject: LU-6179 llite: remove LOCKAHEAD_OLD compatibility
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5315db3f1066619d6effe4f778d2df3ad1ba738f

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/38109
Subject: LU-6179 llite: remove LOCKAHEAD_OLD compatibility
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f6bc909bfda5521454631a4985648e07c63137ee

[Peter Jones]

Landed for 2.11

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/13564/
Subject: LU-6179 llite: Implement ladvise lockahead
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a8dcf372f430c308d3e96fb506563068d0a80c2d

[Patrick Farrell (Inactive)]

Oh. Hm. No - It's skipping some of the tests. Sorry about that, thanks for pointing it out. Some development stuff I was doing escaped in to what I pushed upstream... I'll fix that when I rebase to merge.

[Mikhail Pershin]

Patrick, the new test 255c in sanity.sh reports the following:

== sanity test 255c: suite of ladvise lockahead tests ================================================ 04:54:04 (1495688044) Starting test test10 at 1495688045
Finishing test test10 at 1495688045
Starting test test20 at 1495688045
cannot give advice: Invalid argument (22)
cannot give advice: Invalid argument (22)
cannot give advice: Invalid argument (22)
cannot give advice: Invalid argument (22)
cannot give advice: Invalid argument (22)
cannot give advice: Invalid argument (22)
Finishing test test20 at 1495688045

Is that expected or test isn't working properly?

This is from the latest test results https://testing.hpdd.intel.com/sub_tests/38e1f10a-4129-11e7-91f4-5254006e85c2

 

[Cong Xu (Inactive)]

Hi Patrick,

In the very simple test you suggested, the lock count does increase from 0 to 4010, so the lock ahead works well.
Yes, a faster OST should show more benefits.

[Patrick Farrell (Inactive)]

Huh, OK! That's a clever way to show it.

A faster OST will show much larger benefits, of course.