[LU-7333] nfs modprobe failure on sles11sp3 Created: 23/Oct/15  Updated: 23/Oct/15  Resolved: 23/Oct/15

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Bob Glossman (Inactive) Assignee: WC Triage
Resolution: Not a Bug Votes: 0
Labels: None

Issue Links:
Related
is related to LU-7200 kernel update [SLES11 SP3 3.0.101-0.4... Resolved
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

derived from LU-7200

seeing failure like this on every boot of MDS:

02:33:55:[    1.798369] modprobe: page allocation failure: order:4, mode:0xd0
02:33:55:[    1.799381] Pid: 409, comm: modprobe Tainted: G           E   3.0.101-0.47.67_lustre.g71918e8-default #1
02:33:55:[    1.801128] Call Trace:
02:33:55:[    1.801750]  [<ffffffff81004b95>] dump_trace+0x75/0x300
02:33:55:[    1.802653]  [<ffffffff81465033>] dump_stack+0x69/0x6f
02:33:55:[    1.803551]  [<ffffffff811028d6>] warn_alloc_failed+0xc6/0x170
02:33:55:[    1.804574]  [<ffffffff811044e1>] __alloc_pages_slowpath+0x561/0x7f0
02:33:55:[    1.805603]  [<ffffffff81104959>] __alloc_pages_nodemask+0x1e9/0x200
02:33:55:[    1.806622]  [<ffffffff81146e43>] kmem_getpages+0x53/0x180
02:33:55:[    1.807563]  [<ffffffff81147c56>] fallback_alloc+0x196/0x270
02:33:55:[    1.808524]  [<ffffffff81148847>] kmem_cache_alloc+0x207/0x2a0
02:33:55:[    1.809491]  [<ffffffff8114acb7>] kmem_cache_create+0x297/0x540
02:33:55:[    1.810472]  [<ffffffffa033a316>] nfs_init_writepagecache+0x1f/0xd09 [nfs]
02:33:55:[    1.811563]  [<ffffffffa033a185>] init_nfs_fs+0xe5/0x138 [nfs]
02:33:55:[    1.812545]  [<ffffffff810001cb>] do_one_initcall+0x3b/0x180
02:33:55:[    1.813496]  [<ffffffff810a2e2f>] sys_init_module+0xcf/0x240
02:33:55:[    1.814458]  [<ffffffff8146fe72>] system_call_fastpath+0x16/0x1b
02:33:55:[    1.815442]  [<00007f17535abd5a>] 0x7f17535abd59
02:33:55:[    1.816299] Mem-Info:
02:33:55:[    1.816916] Node 0 DMA per-cpu:
02:33:55:[    1.817655] CPU    0: hi:    0, btch:   1 usd:   0
02:33:55:[    1.818518] Node 0 DMA32 per-cpu:
02:33:55:[    1.819303] CPU    0: hi:   42, btch:   7 usd:   0
02:33:55:[    1.820178] active_anon:1395 inactive_anon:6 isolated_anon:0
02:33:55:[    1.820179]  active_file:3865 inactive_file:3893 isolated_file:0
02:33:55:[    1.820179]  unevictable:9609 dirty:0 writeback:0 unstable:0
02:33:55:[    1.820180]  free:893 slab_reclaimable:893 slab_unreclaimable:4198
02:33:55:[    1.820181]  mapped:1160 shmem:8 pagetables:356 bounce:0
02:34:16:[    1.824950] Node 0 DMA free:460kB min:0kB low:0kB high:0kB active_anon:0kB inactive_anon:0kB active_file:72kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:308kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
02:34:16:[    1.830684] lowmem_reserve[]: 0 115 115 115
02:34:17:[    1.831764] Node 0 DMA32 free:3112kB min:1372kB low:1712kB high:2056kB active_anon:5580kB inactive_anon:24kB active_file:15388kB inactive_file:15572kB unevictable:38436kB isolated(anon):0kB isolated(file):0kB present:118392kB mlocked:8208kB dirty:0kB writeback:0kB mapped:4640kB shmem:32kB slab_reclaimable:3572kB slab_unreclaimable:16792kB kernel_stack:568kB pagetables:1424kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
02:34:17:[    1.837891] lowmem_reserve[]: 0 0 0 0
02:34:17:[    1.838916] Node 0 DMA: 3*4kB 0*8kB 0*16kB 0*32kB 3*64kB 2*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 460kB
02:34:17:[    1.841450] Node 0 DMA32: 142*4kB 126*8kB 54*16kB 17*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3112kB
02:34:17:[    1.844017] 15327 total pagecache pages
02:34:17:[    1.844773] 0 pages in swap cache
02:34:17:[    1.845496] Swap cache stats: add 0, delete 0, find 0/0
02:34:17:[    1.846396] Free swap  = 0kB
02:34:17:[    1.847064] Total swap = 0kB
02:34:17:[    1.847719] 29675 pages RAM
02:34:17:[    1.848471] FS-Cache: Netfs 'nfs' unregistered from caching
02:34:17:modprobe: FATAL: Error inserting nfs (/lib/modules/3.0.101-0.47.67_lustre.g71918e8-default/kernel/fs/nfs/nfs.ko): Cannot allocate memory

It doesn't appear fatal. Execution proceeds after these events. Sure to block use of nfs. I believe it's common in many sites and test environments to mount nfs volumes to all servers for common logging or sharing low data rate info.



 Comments   
Comment by Bob Glossman (Inactive) [ 23/Oct/15 ]

seen not just on MDS. also seen on OSS.

Comment by Andreas Dilger [ 23/Oct/15 ]

This appears to be a problem in the NFS code and not Lustre. It is failing a 64KB allocation and not using GFP_QUIET during allocation. Not sure what we could do about this.

If there are actual failures, or if you think this is caused by Lustre, please reopen.

Comment by Bob Glossman (Inactive) [ 23/Oct/15 ]

fwiw I haven't been able to reproduce this modprobe failure locally. Can't make it happen with a pure untouched and unpatched sles11sp3 kernel or a lustre patched sles11sp3 kernel of this version.

Generated at Sat Feb 10 02:07:59 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.