[LU-5426] Add more controls for open file cache Created: 29/Jul/14  Updated: 27/May/20  Resolved: 27/May/20

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.7.0
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Oleg Drokin Assignee: WC Triage
Resolution: Duplicate Votes: 0
Labels: None

Issue Links:
Duplicate
duplicates LU-10948 client cache open lock after N opens Open
Related
is related to LU-7915 investigate heuristics for SPARK clie... Open
is related to LU-4367 unlink performance regression on lust... Resolved
Rank (Obsolete): 15099

 Description   

In comments to patch http://review.whamcloud.com/#/c/11062/ for LU-4367 John Hammond made a compelling argument that aside from NFS there are other usecases where file caching makes sense.
For example for file exec, while (as demonstrated) running the same file again and again might or might not be frequent, system libraries are definitely going to be opened and closed all the time should they happen to be on Lustre fs.

As such it probably makes sense to have an option (client side? mds side?) to control if we want to always cache files open for execution (easy to tell by open mode).

I imagine there might be compelling usecases for when the same file would need to be opened for read or write all the time on the same client too, though I cannot think of any.
If so, we might also want a client-side control of "always cache open file handles" setting on a per-client basis.

All of the above is relatively easy to implement, but we probably need to discuss first what makes the most sense keeping in mind that always doing the caching is pretty expensive at least for now.



 Comments   
Comment by Andreas Dilger [ 29/Jul/14 ]

I think it is possible to use FMODE_EXEC to detect the executable case, and mmap(PROT_EXEC) to detect the library case. Having a counter for the opens on the client inode, or maybe a per-filesystem "average number of opens per inode" counter would also allow it to try and get the open lock.

I'm not a big fan of having a tunable setting that needs to be adjusted by the user, since that will just be one more thing that users do not know about. Any optimization in this area should be an automatically adjusting heuristic first, and only if we think there is a big risk of performance regression due to the heuristic should a tunable be implemented.

Comment by Andreas Dilger [ 27/May/20 ]

Closing this as a duplicate of LU-10948, which has a patch to implement this functionality.

Generated at Sat Feb 10 01:51:23 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.