[LU-1501] Kernel panic - not syncing: Cannot find space for the kernel page tables

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 1.8.7
• Type: Bug
• Priority: Minor
• Reporter: Minoru Hamakawa
• Assignee: Zhenyu Xu
• Resolution: Fixed
• Labels: client

Environment

Clients: SGI UV1000
kernel: 2.6.32-131.0.15.el6.x86_64
lustre: 1.8.7 wc1

• Severity: 3
• Epic: client

Description

Hello,

One of our, DDN Japan, customer faced kernel panic with RIP in "RIP [<ffffffffa07a32e3>] ll_readahead+0x43/0x1d20 [lustre]".
Attached is the console log of this client.

Could you check if this is Kernel/BIOS issue or Lustre issue?

Many thanks,
Minoru@DDN Japan

Comments

[Peter Jones]

Bobijam

Could you please help with this one?

Thanks

Peter

[Shuichi Ihara (Inactive)]

Bobijam,

Could you please have a look at the log files and advise us first analysis if the lustre is related?

[Zhenyu Xu]

what does the fat_panic_log come about? is it the same client? that part is not a lustre issue.

Can you get the crash dump of the node?

[Minoru Hamakawa]

Thank you, Bobijam,

Yes. it's same client. And both files are console output of that client.
Will ask about crash dump and time relation between two files.

Many thanks,
Minoru

[Minoru Hamakawa]

Hi Bojijam,

I upload whole console log and crash dump under /upload/LU-1501.
Could you look at this?
We can see there are lots of mce log, two hours ago of the panic..

Many thanks,
Minoru

[Zhenyu Xu]

the console.log uploaded under /upload/LU-1501 shows the same kernel error as fat_panic.log showed.

I'm afraid the uploaded "uvdmp_26_2012_6_1_17_22" is not what I want, it says "uvdmp - UV Hub and NL5 Router MMR Capture Utility - Version 2.05" in it, is it a network captured data?

What I meant is the kernel core dump file with its uncompressed kernel image which we can check what possibly happened when the system crashed.

[Minoru Hamakawa]

I'm sorry I upload it without dump..

[Minoru Hamakawa]

Hi Bojijam,

They couldn't get any kernel dump this time..
Could you check whether the root cause of the panic is related to lustre or not with only console msg?

Many thanks,
Minoru

[Zhenyu Xu]

the log.txt relates to Lustre and the fat_panic_log does not, does the system has memory issue?

[Minoru Hamakawa]

We can see followings in /var/log/messages before that panic happened.
MEMLOG[44377]: r001i23b07 P0D-DIMM1A has a failed DRAM and must be replaced soon. Exposure to Uncorrected Error is high

[Zhenyu Xu]

I'd say it's mostly like a hardware problem in this case (memory failure).

[Minoru Hamakawa]

Agreeded. If they face same panic without hardware error, I'll open this again.
So please close this.
Many thanks.

[Peter Jones]

ok thanks!