[LU-2748] OSD uses kmalloc with high order to allocate a keys

• Type: Technical task
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.4.0
• Affects Version/s: Lustre 2.4.0
• Environment: maloo
• Assignee: Alex Zhuravlev
• Reporter: Alexey Lyashkov

Description

05:34:20:ll_ost_io00_029: page allocation failure. order:8, mode:0x50
05:34:20:Pid: 13919, comm: ll_ost_io00_029 Not tainted 2.6.32-279.19.1.el6_lustre.gc7121e9.x86_64 #1
05:34:20:Call Trace:
05:34:20: [<ffffffff811231ff>] ? __alloc_pages_nodemask+0x77f/0x940
05:34:20: [<ffffffff8115d1a2>] ? kmem_getpages+0x62/0x170
05:34:20: [<ffffffff8115ddba>] ? fallback_alloc+0x1ba/0x270
05:34:20: [<ffffffff8115d80f>] ? cache_grow+0x2cf/0x320
05:34:20: [<ffffffff8115db39>] ? ____cache_alloc_node+0x99/0x160
05:34:20: [<ffffffffa04d8b60>] ? cfs_alloc+0x30/0x60 [libcfs]
05:34:20: [<ffffffff8115e909>] ? __kmalloc+0x189/0x220
05:34:20: [<ffffffffa04d8b60>] ? cfs_alloc+0x30/0x60 [libcfs]
05:34:20: [<ffffffffa0d1c8ae>] ? osd_key_init+0x1e/0x670 [osd_ldiskfs]
05:34:20: [<ffffffffa06643ef>] ? keys_fill+0x6f/0x190 [obdclass]
05:34:20: [<ffffffffa066843b>] ? lu_context_init+0xab/0x260 [obdclass]
05:34:20: [<ffffffffa07f78b3>] ? ptlrpc_main+0x203/0x1870 [ptlrpc]
05:34:20: [<ffffffff810097cc>] ? __switch_to+0x1ac/0x320
05:34:20: [<ffffffffa07f76b0>] ? ptlrpc_main+0x0/0x1870 [ptlrpc]
05:34:20: [<ffffffff8100c0ca>] ? child_rip+0xa/0x20
05:34:20: [<ffffffffa07f76b0>] ? ptlrpc_main+0x0/0x1870 [ptlrpc]
05:34:20: [<ffffffffa07f76b0>] ? ptlrpc_main+0x0/0x1870

instead of kmalloc, kmem_cache_alloc should be used to allocate and effectively handle key's.

Issue Links

• is duplicated by: LU-2964 lustre build fails on sles11sp2
• is related to: LU-2790 Failure to allocated osd keys leads to ofd_intent_policy()) ASSERTION( res_lvb != ((void *)0) ) failed
• is related to: LU-2818 Failure on test suite parallel-scale-nfsv4 test_compilebench: (mdt_lvb.c:126:mdt_lvbo_fill()) ASSERTION( rc == 0 ) failed
• is related to: LU-2791 Stuck client on server OOM/lost message
• is related to: LU-2790 Failure to allocated osd keys leads to ofd_intent_policy()) ASSERTION( res_lvb != ((void *)0) ) failed
• is related to: LU-1431 Support for larger than 1MB sequential I/O RPCs

Comments

[Andreas Dilger]

The http://review.whamcloud.com/5444 patch was landed for 2.4.0, and http://review.whamcloud.com/5323 is no longer needed.

[Peter Jones]

Oleg, can your http://review.whamcloud.com/#change,5323 "band aid" patch be abandoned?

[Alex Zhuravlev]

should be resolved now?

[James A Simmons]

Patch to fix this is at http://review.whamcloud.com/#change,5708

[James A Simmons]

Patch to fix this is at http://review.whamcloud.com/#change,5708

[James A Simmons]

I found a bug in http://review.whamcloud.com/#change,5444.

The rhel6.3 ldiskfs patch ext4-map_inode_page-2.6.18.patch defines ext4_map_inode_page function has having a integer array created need for it function but in fsfilt_ext3.c use call out to this function with

extern int ext3_map_inode_page(struct inode *inode, struct page *page,
unsigned long *blocks, int create);

This will crash in a very painful way.

[Peter Jones]

As per Oleg, dropping priority for remaining work

[Alex Zhuravlev]

http://review.whamcloud.com/#change,5444

[Oleg Drokin]

Bandaid patch to just change the allocation to OBD_ALLOC_LARGE is in http://review.whamcloud.com/5323, this seems to greatly help in my testing.

[Andreas Dilger]

Probably dr_blocks[] can be reduced in size as well, since we will normally have 4kB blocksize instead of 1kB blocksize, so we don't need a buffer_head for every 1kB of space.