[LU-14185] alloc fails for OSP obj xattr cache

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0
• Severity: 3
• Assignee: Vitaly Fertman
• Reporter: Vitaly Fertman

Description

Nov 25 05:34:29 snx11205n002 kernel: mdt02_002: page allocation failure: order:5, mode:0x10c050
Nov 25 05:34:29 snx11205n002 kernel: CPU: 6 PID: 40695 Comm: mdt02_002 Kdump: loaded Tainted: P           OE  ------------   3.10.0-957.1.3957.1.3.x3.4.40.x86_64 #1
Nov 25 05:34:29 snx11205n002 kernel: Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0021.032120170601 03/21/2017
Nov 25 05:34:29 snx11205n002 kernel: Call Trace:
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffb0563e41>] dump_stack+0x19/0x1b
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffaffbcab0>] warn_alloc_failed+0x110/0x180
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffb055f44e>] __alloc_pages_slowpath+0x6b6/0x724
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffaffc1115>] __alloc_pages_nodemask+0x405/0x420
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffb000df78>] alloc_pages_current+0x98/0x110
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffaffbb85e>] __get_free_pages+0xe/0x40
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffb001960e>] kmalloc_order_trace+0x2e/0xa0
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffb001d651>] __kmalloc+0x211/0x230
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1bbc1f2>] osp_oac_xattr_find_or_add+0x72/0x270 [osp]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1bc0a99>] osp_xattr_get+0xd29/0x1140 [osp]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc199f181>] lod_xattr_get+0xf1/0x770 [lod]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1a30a7c>] __mdd_permission_internal+0x71c/0x9a0 [mdd]
Nov 25 05:34:29 snx11205n007 kernel: _base_display_event_data: 14 callbacks suppressed
Nov 25 05:34:29 snx11205n007 kernel: mpt3sas_cm0: SAS Broadcast Primitive
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1a0a65f>] __mdd_lookup.isra.20+0x19f/0x440 [mdd]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1a0a9af>] mdd_lookup+0xaf/0x170 [mdd]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1aa9152>] mdt_lookup_version_check+0x72/0x2c0 [mdt]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1aaf81e>] mdt_create+0x5ee/0xf80 [mdt]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1ab031b>] mdt_reint_create+0x16b/0x360 [mdt]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1ab58f3>] mdt_reint_rec+0x83/0x210 [mdt]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1a8f483>] mdt_reint_internal+0x703/0xae0 [mdt]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1a9a987>] mdt_reint+0x67/0x140 [mdt]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1698d4a>] tgt_request_handle+0x96a/0x1680 [ptlrpc]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc11ae117>] ? libcfs_debug_msg+0x57/0x80 [libcfs]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc163e586>] ptlrpc_server_handle_request+0x256/0xb10 [ptlrpc]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc11ae117>] ? libcfs_debug_msg+0x57/0x80 [libcfs]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc164225a>] ptlrpc_main+0xcca/0x1ca0 [ptlrpc]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffc1641590>] ? ptlrpc_register_service+0x1010/0x1010 [ptlrpc]
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffafec1c31>] kthread+0xd1/0xe0
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffafec1b60>] ? insert_kthread_work+0x40/0x40
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffb0576c1d>] ret_from_fork_nospec_begin+0x7/0x21
Nov 25 05:34:29 snx11205n002 kernel:  [<ffffffffafec1b60>] ? insert_kthread_work+0x40/0x40
Nov 25 05:34:29 snx11205n002 kernel: Mem-Info:
Nov 25 05:34:29 snx11205n002 kernel: active_anon:339349 inactive_anon:159427 isolated_anon:0
 active_file:18004076 inactive_file:11307082 isolated_file:0
 unevictable:17581 dirty:76 writeback:0 unstable:0
 slab_reclaimable:947311 slab_unreclaimable:883257
 mapped:21564 shmem:325922 pagetables:7015 bounce:0
 free:183879 free_pcp:29 free_cma:0
Nov 25 05:34:29 snx11205n002 kernel: Node 1 Normal free:236112kB min:45284kB low:56604kB high:67924kB active_anon:330076kB inactive_anon:216040kB active_file:36659868kB inactive_file:23849764kB unevictable:64476kB isolated(anon):0kB isolated(file):0kB present:67108864kB managed:66029160kB mlocked:60824kB dirty:224kB writeback:0kB mapped:50528kB shmem:353876kB slab_reclaimable:1897888kB slab_unreclaimable:1567696kB kernel_stack:11888kB pagetables:6412kB unstable:0kB bounce:0kB free_pcp:116kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Nov 25 05:34:30 snx11205n006 kernel[15415]: Last message 'mpt3sas_cm0: SAS Bro' repeated 7 times, suppressed by syslog-ng on snx11205n006
Nov 25 05:34:30 snx11205n002 kernel: lowmem_reserve[]: 0 0 0 0
Nov 25 05:34:30 snx11205n007 kernel[13379]: Last message 'mpt3sas_cm0: SAS Bro' repeated 7 times, suppressed by syslog-ng on snx11205n007
Nov 25 05:34:30 snx11205n002 kernel: Node 1 Normal: 25903*4kB (UEM) 10038*8kB (UEM) 2590*16kB (UEM) 117*32kB (EM) 11*64kB (UM) 3*128kB (UM) 2*256kB (U) 1*512kB (U) 0*1024kB 0*2048kB 0*4096kB = 231212kB
Nov 25 05:34:30 snx11205n002 kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Nov 25 05:34:30 snx11205n002 kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Nov 25 05:34:30 snx11205n002 kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Nov 25 05:34:30 snx11205n002 kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Nov 25 05:34:30 snx11205n002 kernel: 29641527 total pagecache pages
Nov 25 05:34:30 snx11205n002 kernel: 1436 pages in swap cache
Nov 25 05:34:30 snx11205n002 kernel: Swap cache stats: add 24890, delete 23454, find 2998/3262
Nov 25 05:34:30 snx11205n002 kernel: Free swap  = 16685244kB
Nov 25 05:34:30 snx11205n002 kernel: Total swap = 16777212kB
Nov 25 05:34:30 snx11205n002 kernel: 33529746 pages RAM
Nov 25 05:34:30 snx11205n002 kernel: 0 pages HighMem/MovableOnly
Nov 25 05:34:30 snx11205n002 kernel: 683327 pages reserved
Nov 25 05:34:30 snx11205n002 kernel: Lustre: 40695:0:(osp_object.c:1090:osp_xattr_get()) snx11205-MDT0001-osp-MDT0000: Fail to add xattr (system.posix_acl_access) to cache for [0x240077641:0x23ba:0x0] (1): rc = -12

osp_oac_xattr_find_or_add tries to allocate XATTR_SIZE_MAX