In comments to patch http://review.whamcloud.com/#/c/11062/ for LU-4367 John Hammond made a compelling argument that aside from NFS there are other usecases where file caching makes sense.
For example for file exec, while (as demonstrated) running the same file again and again might or might not be frequent, system libraries are definitely going to be opened and closed all the time should they happen to be on Lustre fs.

As such it probably makes sense to have an option (client side? mds side?) to control if we want to always cache files open for execution (easy to tell by open mode).

I imagine there might be compelling usecases for when the same file would need to be opened for read or write all the time on the same client too, though I cannot think of any.
If so, we might also want a client-side control of "always cache open file handles" setting on a per-client basis.

All of the above is relatively easy to implement, but we probably need to discuss first what makes the most sense keeping in mind that always doing the caching is pretty expensive at least for now.