[LU-5232] cache directory contents on file descriptor on lock revocation Created: 19/Jun/14  Updated: 01/May/23  Resolved: 01/May/23

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.6.0
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Andreas Dilger Assignee: WC Triage
Resolution: Duplicate Votes: 0
Labels: medium, performance

Issue Links:
Duplicate
is duplicated by LU-3308 large readdir chunk size slows unlink... Reopened
Related
is related to LU-4906 rm -rf triggers too much MDS_READPAGE Resolved
is related to LU-3308 large readdir chunk size slows unlink... Reopened
Rank (Obsolete): 14581

 Description   

In LU-4906 the code was changed to read directory entries one page at a time instead of one entry at a time. This improves performance with "rm -r" workloads where entries are being deleted on the MDS and the DLM lock on the directory is being revoked repeatedly, but the client needs to keep fetching it to continue the readdir.

According to SuSv2 it is permissible for readdir() to not be 100% consistent if entries are being added or deleted while readdir() is ongoing:
http://pubs.opengroup.org/onlinepubs/007908799/xsh/readdir.html

If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified.

The readdir() function may buffer several directory entries per actual read operation; readdir() marks for update the st_atime field of the directory each time the directory is actually read.

So in summary, it is possible to cache the readdir() contents on the client, even after the DLM lock is revoked so long as it is only cached on the open file descriptor, and is invalidated upon file close or rewinddir().



 Comments   
Comment by Andreas Dilger [ 01/May/23 ]

Same issue as LU-3308

Generated at Sat Feb 10 01:49:40 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.