[LU-10967] MDT page cache management improvements

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.10.3
• Environment: Relevant to any configuration where critical MDT metadata may not fit in memory or may get pushed out of DRAM cache due to other IO.
• Assignee: Peter Jones
• Reporter: Nathan Dauchy (Inactive)

Description

Opening this as somewhat of a catch all for improvements to MDT memory management, either through pinning or making more selective use of the page cache.  This will be important for at least two cases...
A) Data On MDT
B) Combining OST and MDT on a single server
MDT performance historically relied on "read mostly" workload to backing storage, with much of the important and frequently used metatdata in DRAM cache.  With increase in data blocks from either case above, that could quickly get pushed out of cache and thereby slow down metadata ops.

It may not be practical to try to pin all MDT "real metadata" info in memory, but some features to perhaps release DoM from the page cache faster, or refresh the MDT (and OST) bitmaps and inode tables, would be helpful.

Issue Links

• is related to: LU-15 strange slow IO messages and bad performance
• is related to: LU-3631 file stats are slow on filesystem start up
• is related to: LU-1885 Lustre should be able to limit amount of memory used for read and write caches
• is related to: LU-10946 add an interface to load ldiskfs block bitmaps