Details
-
Bug
-
Resolution: Fixed
-
Major
-
Lustre 2.6.0, Lustre 2.4.2, Lustre 2.5.3
-
3
-
14237
Description
For OSDs like ZFS to perform optimally it's import that they be allowed to manage their own cache. This maximizes the likelyhood that the ARC will prefetch and cache the right buffers. In the existing ZFS OSD code a cached LU object pins buffers in the ARC preventing them from being dropped. As the LU cache grows it can consume the entire ARC preventing buffers for other objects, such as the OIs, from being cached and severely impacting the performance for FID lookups.
The proposed patch addresses this by limiting the size of the lu_cache but alternate approaches are welcome. We are carrying this patch in LLNLs tree and it does help considerably.
Attachments
Issue Links
- is related to
-
LU-5240 Test failure sanity test_69: write error
-
- Resolved
-
My knowledge of the Lustre server stack is very limited, so I'm not sure whether it's feasible or not. But here's my thoughts:
1. Get rid of the LRU completely. Objects are freed once the last reference is dropped. Then it'd be equivalent to the ZPL way of holding on to DMU objects/buffers only for the duration of system calls. This also gives the ARC the freedom to decide which buffers to keep or evict. After all, the ARC is supposed to do a better job than a simple LRU.
2. When osd-zfs has the knowledge that certain objects are frequently used or will be used soon, hold references to those objects proactively. For example:
This way ARC has the freedom it needs, and osd-zfs also contributes when it knows better what to cache. It should be able to handle to case Alex outlined where a client accesses a directory exclusively, because the queued RPCs will keep objects used by the current RPC in the cache.