[LU-10181] DoM performance optimization

• Type: Technical task
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.12.0
• Labels: DoM DoM2
• Assignee: Mikhail Pershin
• Reporter: Mikhail Pershin

Description

Various optimization for better DoM performance. So far they are:

• glimpse ahead. Issue glimpse from MDT in advance and return size on client along with other attributes.
• read-on-open. Read file data at OPEN and return to the client
• lock bits convert instead of cancel. It is separate ticket because it is useful for all files, not just DoM.

Issue Links

• is related to: LU-13136 (layout.c:2121:__req_capsule_get()) @@@ Wrong buffer for field 'niobuf_inline' (7 of 7) in format 'LDLM_INTENT_OPEN', 0 vs. 0 (server)
• is related to: LU-18987 Strange LASSERT in mdt_dom_read_on_open()
• is related to: LU-3285 Data on MDT

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/23011/
Subject: LU-10181 mdt: read on open for DoM files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 13372d6c243ccfc1a0886469f3c0dc048783ed47

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/31750/
Subject: LU-10181 mds: init cpt params for mdt IO service
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3d29ce84a51bcb9593a3d9bca3c570b36b566623

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31768
Subject: LU-10181 mdt: take discard lock at cleanup stage
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 283521ae6fb612598552a56271d11d442b7b2e5b

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31750
Subject: LU-10181 mds: init cpt params for mdt IO service
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 23497102e83b05c1926b9dff652d6fcb299bfea1

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31749
Subject: LU-10181 test: add compilebench and DNE tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 842a88e9adc0679f8f82b8c18af57385639c1ecb

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/30059/
Subject: LU-10181 tests: add FIO as test for DOM
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 84099c2135e8b11ffd2688a4b4c2529e6e60468f

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/29968/
Subject: LU-10181 mdt: high-priority request handling for DOM
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 9e338ee25609d9bbcc359c37012e6868dd6be5d3

[Gerrit Updater]

Mike Pershin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31271
Subject: LU-10181 test: debug tests
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 30ffbccb384ed29e3b02dc9433213127e5e87dfd

[Mikhail Pershin]

I've done quick tests with the patch, on my local VM. There are some numbers:
1. time grep -R "something" on linux source dir

DoM (Read-on-Open)   : real 1m6.677s  user 0m1.107s sys 0m35.430s
                       real 1m8.711s  user 0m1.129s sys 0m35.416s
                       real 1m4.229s  user 0m1.071s sys 0m34.549s
DoM (no Read-on-Open): real 1m9.945s  user 0m0.926s sys 0m35.330s
                       real 1m10.561s user 0m0.930s sys 0m34.905s
                       real 1m11.278s user 0m0.871s sys 0m35.280s
OST files (2 stripes): real 1m23.753s user 0m1.033s sys 0m37.412s
                       real 1m27.987s user 0m1.032s sys 0m39.327s
                       real 1m20.697s user 0m0.993s sys 0m35.494s

RPC stats shows there are only 593 READ RPCs for about 47697 files with read-on-open, but resulting time is less not much, probably the difference will be seen better on real system, not VM.

2. dbench 4

DoM (Read-on-Open)   : Throughput 42.459 MB/sec  4 clients  4 procs  max_latency=1224.798 ms
DoM (no Read-on-Open): Throughput 37.2351 MB/sec  4 clients  4 procs  max_latency=1163.293 ms
OST files (2 stripes): Throughput 28.0458 MB/sec  4 clients  4 procs  max_latency=2027.644 ms

Dbench shows clear difference in results for DoM in general and Read-on-Open

 

[Joseph Gmitter (Inactive)]

read-on-open patches:
https://review.whamcloud.com/#/c/23011/