[LU-14139] batched statahead processing

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Assignee: Qian Yingjin
• Reporter: Qian Yingjin

Description

Batched metadata processing may get a big performance boost.
Batched statahead can also increase the performance for directory listing operations such as ls.

Issue Links

• is related to: LU-16365 cached 'ls -l' is slow
• is related to: LU-17783 Wrong buffer for field 'batch_update_reply' in format 'MDS_BATCH' from statahead for old server
• is related to: LU-8582 Interop: master<->b2_8 - sanity test_255a: test failed to respond and timed out
• is related to: LU-10334 Ubuntu1604 client sanity-103a: FAIL: ACL misc test failed
• is related to: LU-14868 sanity: all subtests pass but test suite fails with TIMEOUT
• is related to: LU-16247 do open()+readahead for DoM files as batched operations
• is related to: LU-16355 batch dirty buffered write of small files
• is related to: LU-23 Improve performance for traversing large directory with readdir+
• is related to: LU-10280 DoM cross-file open+readahead via statahead
• is related to: LU-14380 Make statahead better support Breadth First Search (BFS) or Depth First Search (DFS)
• is related to: LU-14138 Move more members in PTLRPC request into pill
• is related to: LU-14361 Add support for statahead pattern with regularized file name
• is related to: LU-15975 Statahead_V1 Features
• is related to: LU-16139 statahead: avoid new RPC and long wait in the statahead interpret callback
• is related to: LU-10938 Metadata writeback cache support
• is related to: LU-13045 WBC2: batch metadata update
• is related to: LU-16096 recovery: handle compatibility during upgrade for new replay data format
• is related to: LU-14361 Add support for statahead pattern with regularized file name
• is related to: LU-14393 Batching Processing Framework for Lustre
• is related to: LU-15550 WBC: retry the batched RPC when the reply buffer is overflowed
• is related to: LU-15562 statahead: using try lock for batched statahead to avoid deadlock with rename and migrate operations

Comments

[Peter Jones]

Landed for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/41220/
Subject: LU-14139 statahead: add test for batch statahead processing
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cd1e7bdd903ada4467064eaf8613ec62a358fa1c

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/43707/
Subject: LU-14139 ptlrpc: grow PtlRPC properly when prepare sub request
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5a2dfd36f9c2b6c10ab7ba44b0e9e86372623fde

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40945/
Subject: LU-14139 ptlrpc: grow reply buffer properly for batch request
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5edb883b44ac707528ce2c0bc812d65b9ffb4a50

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40943/
Subject: LU-14139 statahead: add stats for batch RPC requests
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a20f25d24b5f0ce7b5e77f7c596bffd0450cbdae

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/40720/
Subject: LU-14139 statahead: batched statahead processing
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4435d0121f72aac3ad01c98a33b265a496359890

[Andreas Dilger]

Is there a patch that converts ldlm to use rhashtable?

[Qian Yingjin]

From the attaching FG, lots of time costs are:
__ldlm_handle2lock () ->class_handle2object()
ldlm_resource_get() >cfs_hash_bd_lookup_intent()>ldlm_res_hop_keycmp()

Each of them costs 5% for data and metadata DLM lock matching on the client side. All takes about 20% of the total sample...
It seems that the hash for looking up the lock handle and resource take lots of time.
And ldlm_res_hop_keycmp() reaches 5%, it means there are lots of elements in the same bucket of the hash table. We should increase the hash table and use the resizable hash table.

For class_handle2object for locks, we should maintain the lock handle per target (osc/mdc) lock namespace, not shared a single global hash table for handle on the client or even the server.