[LU-5328] Failure on test suite replay-vbr test_7a: MDS oom

• Type: Bug
• Resolution: Duplicate
• Priority: Minor
• Affects Version/s: Lustre 2.6.0
• Severity: 3
• Assignee: WC Triage
• Reporter: Maloo

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/0048beca-07e1-11e4-9ea6-5254006e85c2.

The sub-test test_7a failed with the following error:

test failed to respond and timed out

04:05:55:Lustre: DEBUG MARKER: e2label /dev/lvm-Role_MDS/P1 2>/dev/null
04:05:55:mdt_out00_001 invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0
04:05:55:mdt_out00_001 cpuset=/ mems_allowed=0
04:05:55:Pid: 4828, comm: mdt_out00_001 Not tainted 2.6.32-431.20.3.el6_lustre.g5a7c614.x86_64 #1
04:05:55:Call Trace:
04:05:55: [<ffffffff810d03d1>] ? cpuset_print_task_mems_allowed+0x91/0xb0
04:05:55: [<ffffffff81122780>] ? dump_header+0x90/0x1b0
04:05:55: [<ffffffff812283cc>] ? security_real_capable_noaudit+0x3c/0x70
04:05:55: [<ffffffff81122c02>] ? oom_kill_process+0x82/0x2a0
04:05:55: [<ffffffff81122afe>] ? select_bad_process+0x9e/0x120
04:05:55: [<ffffffff81123040>] ? out_of_memory+0x220/0x3c0
04:05:55: [<ffffffff8112f95f>] ? __alloc_pages_nodemask+0x89f/0x8d0
04:05:55: [<ffffffff8116e242>] ? kmem_getpages+0x62/0x170
04:05:55: [<ffffffff8116ee5a>] ? fallback_alloc+0x1ba/0x270
04:05:55: [<ffffffff8116e8af>] ? cache_grow+0x2cf/0x320
04:05:55: [<ffffffff8116ebd9>] ? ____cache_alloc_node+0x99/0x160
04:05:55: [<ffffffff8124c7fc>] ? crypto_create_tfm+0x3c/0xe0
04:05:55: [<ffffffff8116f9a9>] ? __kmalloc+0x189/0x220
04:05:55: [<ffffffff8124c7fc>] ? crypto_create_tfm+0x3c/0xe0
04:05:55: [<ffffffff81253198>] ? crypto_init_shash_ops+0x68/0x100
04:05:55: [<ffffffff8124c90a>] ? __crypto_alloc_tfm+0x6a/0x130
04:05:55: [<ffffffff8124d17a>] ? crypto_alloc_base+0x5a/0xb0
04:05:55: [<ffffffffa04a2634>] ? cfs_percpt_unlock+0x24/0xb0 [libcfs]
04:05:55: [<ffffffffa048d217>] ? cfs_crypto_hash_alloc+0x77/0x290 [libcfs]
04:05:55: [<ffffffffa048d8f6>] ? cfs_crypto_hash_digest+0x66/0xf0 [libcfs]
04:05:55: [<ffffffff8116fa2c>] ? __kmalloc+0x20c/0x220
04:05:55: [<ffffffffa0829503>] ? lustre_msg_calc_cksum+0xd3/0x130 [ptlrpc]
04:05:55: [<ffffffffa0863691>] ? null_authorize+0xa1/0x100 [ptlrpc]
04:05:55: [<ffffffffa0852766>] ? sptlrpc_svc_wrap_reply+0x56/0x1c0 [ptlrpc]
04:05:55: [<ffffffffa082192c>] ? ptlrpc_send_reply+0x1fc/0x7f0 [ptlrpc]
04:05:55: [<ffffffffa0838e75>] ? ptlrpc_at_check_timed+0xc05/0x1360 [ptlrpc]
04:05:55: [<ffffffffa08303c9>] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc]
04:05:55: [<ffffffffa083a768>] ? ptlrpc_main+0x1198/0x1980 [ptlrpc]
04:05:55: [<ffffffffa08395d0>] ? ptlrpc_main+0x0/0x1980 [ptlrpc]
04:05:55: [<ffffffff8109abf6>] ? kthread+0x96/0xa0
04:05:55: [<ffffffff8100c20a>] ? child_rip+0xa/0x20
04:05:55: [<ffffffff8109ab60>] ? kthread+0x0/0xa0
04:05:55: [<ffffffff8100c200>] ? child_rip+0x0/0x20
04:05:55:Mem-Info:
04:05:55:Node 0 DMA per-cpu:
04:05:55:CPU    0: hi:    0, btch:   1 usd:   0
04:05:55:CPU    1: hi:    0, btch:   1 usd:   0
04:05:55:Node 0 DMA32 per-cpu:
04:05:55:CPU    0: hi:  186, btch:  31 usd:  28
04:05:55:CPU    1: hi:  186, btch:  31 usd:   0
04:05:55:active_anon:180 inactive_anon:167 isolated_anon:0
04:05:55: active_file:1024 inactive_file:1012 isolated_file:0
04:05:55: unevictable:0 dirty:0 writeback:31 unstable:0
04:05:55: free:13337 slab_reclaimable:1924 slab_unreclaimable:437116
04:05:55: mapped:0 shmem:0 pagetables:466 bounce:0
04:05:55:Node 0 DMA free:8336kB min:332kB low:412kB high:496kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15348kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:7408kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
04:05:55:lowmem_reserve[]: 0 2004 2004 2004
04:05:55:Node 0 DMA32 free:44640kB min:44720kB low:55900kB high:67080kB active_anon:592kB inactive_anon:540kB active_file:4096kB inactive_file:4048kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:2052308kB mlocked:0kB dirty:0kB writeback:124kB mapped:0kB shmem:0kB slab_reclaimable:7696kB slab_unreclaimable:1741644kB kernel_stack:1608kB pagetables:1864kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
04:05:55:lowmem_reserve[]: 0 0 0 0
04:05:55:Node 0 DMA: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 2*2048kB 1*4096kB = 8324kB
04:05:55:Node 0 DMA32: 1069*4kB 783*8kB 525*16kB 301*32kB 69*64kB 14*128kB 9*256kB 3*512kB 2*1024kB 0*2048kB 1*4096kB = 44764kB
04:05:55:264 total pagecache pages
04:05:55:113 pages in swap cache
04:05:55:Swap cache stats: add 7476, delete 7363, find 76046/76613
04:05:55:Free swap  = 4117516kB
04:05:55:Total swap = 4128760kB
04:05:55:524284 pages RAM
04:05:55:43694 pages reserved
04:05:55:168 pages shared
04:05:55:462234 pages non-shared
04:05:55:[ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
04:05:55:[  367]     0   367     2688        0   1     -17         -1000 udevd
04:05:55:[  994]     0   994    23298       13   1     -17         -1000 auditd
04:05:55:[ 1181]    81  1181     6436        1   1       0             0 dbus-daemon
04:05:55:[ 1196]     0  1196    53901       15   1       0             0 ypbind
04:05:55:[ 1261]     0  1261     1020        0   0       0             0 acpid
04:05:55:[ 1270]    68  1270    10460        1   0       0             0 hald
04:05:55:[ 1271]     0  1271     5081        1   1       0             0 hald-runner
04:05:55:[ 1303]     0  1303     5611        1   1       0             0 hald-addon-inpu
04:05:55:[ 1314]    68  1314     4483        1   1       0             0 hald-addon-acpi
04:05:55:[ 1350]     0  1350    26827        0   0       0             0 rpc.rquotad
04:05:55:[ 1354]     0  1354     5414        0   0       0             0 rpc.mountd
04:05:55:[ 1389]     0  1389     6291        1   0       0             0 rpc.idmapd
04:05:55:[ 1419]   498  1419    57322        1   1       0             0 munged
04:05:55:[ 1434]     0  1434    16651        0   0     -17         -1000 sshd
04:05:55:[ 1442]     0  1442     5545        1   0       0             0 xinetd
04:05:55:[ 1466]     0  1466    22314        0   1       0             0 sendmail
04:05:55:[ 1474]    51  1474    20178        0   0       0             0 sendmail
04:05:55:[ 1496]     0  1496    29325        1   1       0             0 crond
04:05:55:[ 1507]     0  1507     5385        0   0       0             0 atd
04:05:55:[ 1520]     0  1520     1020        1   1       0             0 agetty
04:05:55:[ 1522]     0  1522     1016        1   0       0             0 mingetty
04:05:55:[ 1524]     0  1524     1016        1   0       0             0 mingetty
04:05:55:[ 1526]     0  1526     1016        1   0       0             0 mingetty
04:05:55:[ 1528]     0  1528     1016        1   0       0             0 mingetty
04:05:55:[ 1530]     0  1530     2689        0   1     -17         -1000 udevd
04:05:55:[ 1531]     0  1531     2687        0   0     -17         -1000 udevd
04:05:55:[ 1532]     0  1532     1016        1   0       0             0 mingetty
04:05:55:[ 1534]     0  1534     1016        1   1       0             0 mingetty
04:05:55:[ 2062]    38  2062     7686        4   0       0             0 ntpd

Issue Links

• duplicates: LU-5077 insanity test_1: out of memory on MDT in crypto_create_tfm()