[LU-17276] Use interval tree in flock to improve the scalability

• Type: Question/Request
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0, Lustre 2.17.0
• Severity: 3
• Assignee: Yang Sheng
• Reporter: Yang Sheng

Description

Running a test to exercise flock (N clients requesting a lock on different non-overlapping regions) showed high load on the MDS. There was previously a similar issue with regular extent locks (which are used to control data coherency for regular reads and writes) and adding an extra structure called interval tree reduced the load. Likely the same structure can be used for flock.
 
With a trivial benchmark, single client, local setup, non-overlapping locks:

FLOCKS_TEST 5: SET write 1000 flock(s) took 0.06s 16048.30/sec
FLOCKS_TEST 5: SET write 2000 flock(s) took 0.14s 14526.76/sec
FLOCKS_TEST 5: SET write 5000 flock(s) took 0.60s 8264.82/sec
FLOCKS_TEST 5: SET write 10000 flock(s) took 2.94s 3401.57/sec
FLOCKS_TEST 5: SET write 25000 flock(s) took 36.29s 688.98/sec
FLOCKS_TEST 5: SET write 50000 flock(s) took 281.29s 177.75/sec
FLOCKS_TEST 5: SET write 75000 flock(s) took 661.73s 113.34/sec

Test case in patch https://review.whamcloud.com/53094 "LU-17276 tests: flock scalability test".

Issue Links

• is duplicated by: LU-18439 LLTPfcntl tests are failing during regression IO test
• is related to: LU-17537 Ability to set localflock on a per-process/file basis
• is related to: LU-17502 distribute flock locking across multiple MDS nodes
• is related to: LU-11085 Replace Lustre interval tree with kernel one

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57105/
Subject: LU-17276 flock: fix for lock split
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: f785bca0a92e499493fa0702fe7c0fb05e7b51ca

[Gerrit Updater]

"Yang Sheng <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57105
Subject: LU-17276 flock: fix for lock split
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b83ff653394eb87600690d81b54bd18097beeee6

[Peter Jones]

Ah. I was thinking that there was still https://review.whamcloud.com/c/fs/lustre-release/+/53551 but, based on your comment in Gerrit, perhaps that can be abandoned...

[James A Simmons]

last patch landed.

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54364/
Subject: LU-17276 tests: performance test for same range lock
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a863101084654821a7a317f75d13c3a37b48c7e2

[James A Simmons]

One patch left

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/53951/
Subject: LU-17276 ldlm: use interval tree for searching in flock
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ea012dfb2a94060450c75c4f425f2d8b8c4329f5

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/53950/
Subject: LU-17276 ldlm: convert flock locks to linux interval tree.
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 1c635e263f0aaf4827cf69cbd48b7290efc9e085

[Yang Sheng]

Hi, James, looks like it cannot improve the performance from my local test. Maybe we should do more test to make sure that?

[James A Simmons]

Yang is https://review.whamcloud.com/c/fs/lustre-release/+/53495 still needed?