[LU-6842] Seeking the option to hook cl_page LRU up to kernel cache shrinker

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.8.0
• Labels: llnl
• Assignee: Zhenyu Xu
• Reporter: Jinshan Xiong (Inactive)

Description

The purpose is that the client can cache max_cached_mb at maximum on the client side but if the memory is in pressure it should consume less by hooking it up to cache shrinker.

Issue Links

• is related to: LU-5841 Lustre 2.4.2 MDS, hitting OOM errors

Comments

[Andrew Perepechko]

bobijam,

+#ifndef HAVE_SHRINKER_COUNT
+static int osc_cache_shrink(SHRINKER_ARGS(sc, nr_to_scan, gfp_mask))
+{
+       struct shrink_control scv = {
+               .nr_to_scan = shrink_param(sc, nr_to_scan),
+               .gfp_mask   = shrink_param(sc, gfp_mask)
+       };
+#if !defined(HAVE_SHRINKER_WANT_SHRINK_PTR) && !defined(HAVE_SHRINK_CONTROL)
+       struct shrinker *shrinker = NULL;
+#endif
+
+       (void)osc_cache_shrink_scan(shrinker, &scv);
+
+       return osc_cache_shrink_count(shrinker, &scv);
+}
+#endif

Is there any particular reason to return the value from osc_cache_shrink_count() instead of osc_cache_shrink_scan() itself?

         kswapd0-43    [003] ....  2885.831749: mm_shrink_slab_start: osc_cache_shrink+0x0/0x60 [osc] ffff8800cd67c1c0: objects to shrink 56 gfp_flags GFP_KERNEL pgs_scanned 222 lru_pgs 665761 cache items 645137 delta 430 total_scan 486
         kswapd0-43    [003] d...  2885.831751: r_osc_cache_shrink_scan_0: (osc_cache_shrink+0x36/0x60 [osc] <- osc_cache_shrink_scan) arg1=0x0
         kswapd0-43    [003] d...  2885.831752: r_osc_cache_shrink_0: (shrink_slab+0x15c/0x340 <- osc_cache_shrink) arg1=0x9d811
         kswapd0-43    [003] d...  2885.832371: r_osc_cache_shrink_scan_0: (osc_cache_shrink+0x36/0x60 [osc] <- osc_cache_shrink_scan) arg1=0x80
         kswapd0-43    [003] d...  2885.832374: r_osc_cache_shrink_0: (shrink_slab+0x175/0x340 <- osc_cache_shrink) arg1=0x9d791
         kswapd0-43    [003] d...  2885.832377: r_osc_cache_shrink_scan_0: (osc_cache_shrink+0x36/0x60 [osc] <- osc_cache_shrink_scan) arg1=0x0
         kswapd0-43    [003] d...  2885.832378: r_osc_cache_shrink_0: (shrink_slab+0x15c/0x340 <- osc_cache_shrink) arg1=0x9d791
         kswapd0-43    [003] d...  2885.833002: r_osc_cache_shrink_scan_0: (osc_cache_shrink+0x36/0x60 [osc] <- osc_cache_shrink_scan) arg1=0x80
         kswapd0-43    [003] d...  2885.833004: r_osc_cache_shrink_0: (shrink_slab+0x175/0x340 <- osc_cache_shrink) arg1=0x9d711
         kswapd0-43    [003] d...  2885.833008: r_osc_cache_shrink_scan_0: (osc_cache_shrink+0x36/0x60 [osc] <- osc_cache_shrink_scan) arg1=0x0
         kswapd0-43    [003] d...  2885.833009: r_osc_cache_shrink_0: (shrink_slab+0x15c/0x340 <- osc_cache_shrink) arg1=0x9d711
         kswapd0-43    [003] d...  2885.833569: r_osc_cache_shrink_scan_0: (osc_cache_shrink+0x36/0x60 [osc] <- osc_cache_shrink_scan) arg1=0x80
         kswapd0-43    [003] d...  2885.833571: r_osc_cache_shrink_0: (shrink_slab+0x175/0x340 <- osc_cache_shrink) arg1=0x9d691
         kswapd0-43    [003] ....  2885.833573: mm_shrink_slab_end: osc_cache_shrink+0x0/0x60 [osc] ffff8800cd67c1c0: unused scan count 56 new scan count 102 total_scan 46 last shrinker return val 644753

It seems like in such a scenario, vmscan requested 3 times to scan 128 objects, osc_cache_shrink_scan() reported 3 times that 128 objects were freed. However, the shrinker returned not 0x80;0x80;0x80, but 0x9d791; 0x9d711; 0x9d691, reported as "last shrinker return val 644753".

P.S. arg1 is $retval for osc_cache* retprobes.

[Peter Jones]

Landed for 2.8

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/15630/
Subject: LU-6842 clio: add cl_page LRU shrinker
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 888a3141e72a25bef8daf822325b4295e5a0d5e8

[Cliff White (Inactive)]

Added 2.7.56 results for comparison

[Jinshan Xiong (Inactive)]

memhog is a test program under $LUSTRE/lustre/test. It just allocates a huge amount of memory so that we can investigate if Lustre does well under memory pressure.

[Patrick Farrell (Inactive)]

Cliff - Can you explain the tests results a bit more? What's memhog, for example? Is all the testing with the patch installed? If so, are there non-patched results somewhere to compare to?