[LU-10967] MDT page cache management improvements

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.10.3
• Type: Improvement
• Priority: Minor
• Reporter: Nathan Dauchy (Inactive)
• Assignee: Peter Jones
• Resolution: Unresolved

Environment

Relevant to any configuration where critical MDT metadata may not fit in memory or may get pushed out of DRAM cache due to other IO.

Description

Opening this as somewhat of a catch all for improvements to MDT memory management, either through pinning or making more selective use of the page cache.  This will be important for at least two cases...
A) Data On MDT
B) Combining OST and MDT on a single server
MDT performance historically relied on "read mostly" workload to backing storage, with much of the important and frequently used metatdata in DRAM cache.  With increase in data blocks from either case above, that could quickly get pushed out of cache and thereby slow down metadata ops.

It may not be practical to try to pin all MDT "real metadata" info in memory, but some features to perhaps release DoM from the page cache faster, or refresh the MDT (and OST) bitmaps and inode tables, would be helpful.

Issue Links

• is related to: LU-15 strange slow IO messages and bad perf...
• is related to: LU-3631 file stats are slow on filesystem sta...
• is related to: LU-1885 Lustre should be able to limit amount...
• is related to: LU-10946 add an interface to load ldiskfs bloc...

Comments

[Nathan Dauchy (Inactive)]

The other tickets I could find that are even marginally related to this issue...
LU-1885 : Lustre should be able to limit amount of memory used for read and write caches
LU-2477 : Poor MDS create performance due to ARC cache growth
LU-10946 : add an interface to load ldiskfs block bitmaps

[Peter Jones]

Nathan

Thanks for opening this ticket. It will be good to get a variety of views onto possible approaches here.

Peter

[Andreas Dilger]

The proposal for LU-10946 to pin the block and inode bitmaps would also help for the MDS, though to a lesser extent because the MDT is typically flash so reading them on demand from storage will be much faster than competing with a streaming I/O workload on the OST on spinning disks.

The block bitmaps would need 4KB of RAM per 128MB of MDT space, and the inode bitmaps would double that. For a largish MDTs, say 8TB (=4B files), this would mean 4KB * (8TB/128MB) = 256MB for the block bitmaps, and an additional 256MB for the inode bitmaps, which is not unreasonable. Pinning the inode tables on the MDT would be totally impractical, since they consume (by default) 1/2 of all the MDT space (i.e. you would need 4TB of RAM to pin all of the MDT inode tables).