[LU-13336] Opendir with read intent

• Status: Open
• Project: Lustre
• Type: New Feature
• Priority: Minor
• Reporter: Nathan Rutman
• Assignee: WC Triage
• Resolution: Unresolved

Description

Lustre tree walking performance is fairly poor (~30K files/s) while stat() performance is decent (~200K files/s). The culprit seems to be readdir, which is a series of small file reads from the MDS. (Note readdir isn't tested by mdtest.)

While there are a number of ideas to improve tree walking performance, we think the best potential is to implement opendir with read intent. Opendir should read and return the first page of the directory along with the directory read lock. This can avoid a bunch of RPCs: open, read req, read RDMA.

For small directories (the majority), a single page may well be sufficient to include all dirents, meaning no more RPCs necessary. For larger dirs, the client can begin processing the first page while other pages are delivered from the MDS.

Ref: Cray LUS-7695

Comments

[Andreas Dilger]

I wouldn't be against reading more than a single page. We already have "max_readahead_whole_mb" for files, typically 2MB, so that the whole file is read into client cache on first access. Since network latency is half of the battle, it likely wouldn't be bad to read a whole RPC's worth of pages into the client cache on open.

That said - one case that needs to be determined is whether the read is for readdir(), or whether it is for ioctl(). While I suspect readdir() is overwhelmingly the majority, it would be good if we could distinguish them. In any case, having a tunable to benchmark this and change it in the future is necessary.