[LU-16247] do open()+readahead for DoM files as batched operations

• Status: Open
• Project: Lustre
• Type: New Feature
• Priority: Minor
• Reporter: Qian Yingjin
• Assignee: Qian Yingjin
• Resolution: Unresolved

Description

Lustre currently supported read on open() call for DoM files in single RPC (it will benefit the performance of mdtest read):

https://review.whamcloud.com/23011 "LU-10181 mdt: read on open for DoM files"

It is possible to do open()+readahead for DoM files as batched operations.

But the client needs to know the file names of the open calls in advance and a open/read -ahead mechanism similar to statahead.

This needs the file names of the open calls are predictable such as:

1. Get the file names via readdir() calls;
2. regularized file names such as mdtest.$rank.$i

For each open() + readahead request in the batched RPC,

• the MDS will return MDS_INODELOCK_OPEN ibit lock to the client and cache this lock on the client lock namespace, thus the corresponding file handle protected by this lock can be also validly cached on the client and subsequent predictable open calls can open locally;
• For the readahead data, it will store on the client page cache, and the subsequent read() calls can directly read data from page cache, does not need a extra RPC for file data.

 

Issue Links

• is related to: LU-14139 batched statahead processing

Comments

[Andreas Dilger]

This looks very interesting, and could potentially improve the small file performance significantly for mdtest-hard-read workloads.