[LU-14139] batched statahead processing

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Assignee: Qian Yingjin
• Reporter: Qian Yingjin

Description

Batched metadata processing may get a big performance boost.
Batched statahead can also increase the performance for directory listing operations such as ls.

Issue Links

• is related to: LU-16365 cached 'ls -l' is slow
• is related to: LU-17783 Wrong buffer for field 'batch_update_reply' in format 'MDS_BATCH' from statahead for old server
• is related to: LU-8582 Interop: master<->b2_8 - sanity test_255a: test failed to respond and timed out
• is related to: LU-10334 Ubuntu1604 client sanity-103a: FAIL: ACL misc test failed
• is related to: LU-14868 sanity: all subtests pass but test suite fails with TIMEOUT
• is related to: LU-16247 do open()+readahead for DoM files as batched operations
• is related to: LU-16355 batch dirty buffered write of small files
• is related to: LU-23 Improve performance for traversing large directory with readdir+
• is related to: LU-10280 DoM cross-file open+readahead via statahead
• is related to: LU-14361 Add support for statahead pattern with regularized file name
• is related to: LU-14380 Make statahead better support Breadth First Search (BFS) or Depth First Search (DFS)
• is related to: LU-14138 Move more members in PTLRPC request into pill
• is related to: LU-15975 Statahead_V1 Features
• is related to: LU-16139 statahead: avoid new RPC and long wait in the statahead interpret callback
• is related to: LU-10938 Metadata writeback cache support
• is related to: LU-13045 WBC2: batch metadata update
• is related to: LU-14361 Add support for statahead pattern with regularized file name
• is related to: LU-16096 recovery: handle compatibility during upgrade for new replay data format
• is related to: LU-14393 Batching Processing Framework for Lustre
• is related to: LU-15550 WBC: retry the batched RPC when the reply buffer is overflowed
• is related to: LU-15562 statahead: using try lock for batched statahead to avoid deadlock with rename and migrate operations