[LU-5727] MDS OOMs with 2.5.3 clients and lru_size != 0

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.7.0, Lustre 2.5.4
• Affects Version/s: Lustre 2.5.3
• Labels: llnl
• Severity: 3
• Assignee: Niu Yawei (Inactive)
• Reporter: David Singleton

Description

We have seen mds (admittedly with smallish memory) OOM'ing while testing 2.5.3 whereas there was no problem with 2.5.0. It turns out the problem is that, even though we have lru_size=800 everywhere, the client LDLM lru's are growing huge so that the MDS unreclaimable ldlm slabs fill memory.

It looks like the root cause is the change to ldlm_cancel_aged_policy() in commit 0a6c6fcd46 on the 2.5 branch (LU-4786 osc: to not pick busy pages for ELC) - it has changed the lru_sze != 0 behaviour. Prior to that, the non-lru_resize behaviour (at least through the early_lock_cancel path which is what we see being hit) was effectively

cancel lock if (too many in lru cache || lock unused too long)

In 2.5.3, it's

cancel lock if (too many in lru cache && lock unused too long)

Disabling early_lock_cancel doesn't seem to help.

It might be arguable which of the two behaviours is correct but the lru_size doco suggests the former - the latter makes lru_size != 0 ineffective in practice. It also looks like the change was not actually necessary for LU-4300?

Issue Links

• is related to: LU-6529 Server side lock limits to avoid unnecessary memory exhaustion
• is related to: LU-4300 ptlrpcd threads deadlocked in cl_lock_mutex_get