[LU-5773] obdfilter-survey test 1c: oom occurred on OSS

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.7.0
• Affects Version/s: Lustre 2.7.0, Lustre 2.5.3
• Labels: 22pl MB mq115
• Environment: Lustre build: https://build.hpdd.intel.com/job/lustre-master/2684; Distro/Arch: RHEL6.5/x86_64
• Severity: 3
• Assignee: Niu Yawei (Inactive)
• Reporter: Jian Yu

Description

While running obdfilter-survey test 1c, oom failure occurred on OSS:

21:17:56:Lustre: DEBUG MARKER: == obdfilter-survey test 1c: Object Storage Targets survey, big batch == 02:50:56 (1412823056)
21:17:56:Lustre: DEBUG MARKER: lctl dl | grep obdfilter
21:17:56:Lustre: DEBUG MARKER: /usr/sbin/lctl list_nids | grep tcp | cut -f 1 -d '@'
21:17:56:Lustre: Echo OBD driver; http://www.lustre.org/
21:17:56:hrtimer: interrupt took 7516 ns
21:17:56:lctl invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
21:17:56:lctl cpuset=/ mems_allowed=0
21:17:56:Pid: 19467, comm: lctl Not tainted 2.6.32-431.29.2.el6_lustre.x86_64 #1
21:17:56:Call Trace:
21:17:56: [<ffffffff810d07b1>] ? cpuset_print_task_mems_allowed+0x91/0xb0
21:17:56: [<ffffffff81122b80>] ? dump_header+0x90/0x1b0
21:17:56: [<ffffffff81122cee>] ? check_panic_on_oom+0x4e/0x80
21:17:56: [<ffffffff811233db>] ? out_of_memory+0x1bb/0x3c0
21:17:56: [<ffffffff8112fd5f>] ? __alloc_pages_nodemask+0x89f/0x8d0
21:17:56: [<ffffffff81167dea>] ? alloc_pages_vma+0x9a/0x150
21:17:56: [<ffffffff811499dd>] ? do_wp_page+0xfd/0x920
21:17:56: [<ffffffff8133e4f5>] ? misc_open+0x1d5/0x330
21:17:56: [<ffffffff8114a9fd>] ? handle_pte_fault+0x2cd/0xb00
21:17:56: [<ffffffff8118d495>] ? chrdev_open+0x125/0x230
21:17:56: [<ffffffff811ab840>] ? mntput_no_expire+0x30/0x110
21:17:56: [<ffffffff8118d370>] ? chrdev_open+0x0/0x230
21:17:56: [<ffffffff811863bf>] ? __dentry_open+0x23f/0x360
21:17:56: [<ffffffff812284ef>] ? security_inode_permission+0x1f/0x30
21:17:56: [<ffffffff8114b45a>] ? handle_mm_fault+0x22a/0x300
21:17:56: [<ffffffff8104a8d8>] ? __do_page_fault+0x138/0x480
21:17:56: [<ffffffff8152f25e>] ? do_page_fault+0x3e/0xa0
21:17:56: [<ffffffff8152f25e>] ? do_page_fault+0x3e/0xa0
21:17:56: [<ffffffff8152c615>] ? page_fault+0x25/0x30

Maloo report: https://testing.hpdd.intel.com/test_sets/973e0216-4fcd-11e4-8e65-5254006e85c2

Issue Links

• duplicates: LU-5079 conf-sanity test_47 timeout
• duplicates: LU-3366 Test failure obdfilter-survey, subtest test_1c: oom-killer
• is duplicated by: LU-6004 obdfilter-survey test_2a: obdfilter is missing when running "lctl dl"
• is duplicated by: LU-5920 obdfilter-survey test_1c: OST OOM
• is related to: LU-6064 obdfilter-survey test_1c: test failed to respond and timed out
• is related to: LU-4768 ost-survey hangs on client 2.4