[LU-4013] opening "per_cpu_rebuild" from lprocfs in mc on a freshly mounted Lustre fs causes kernel crash

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.1.0
• Type: Bug
• Priority: Minor
• Reporter: parinay v kondekar (Inactive)
• Assignee: WC Triage
• Resolution: Not a Bug
• Severity: 3

Description

cat of per_cpu_rebuild also results in this,

BUG: unable to handle kernel NULL pointer dereference at 00000000000000a0
IP: [<ffffffffa011c043>] lprocfs_read_helper+0x23/0x120 [lvfs]
PGD 88b5067 PUD 8893067 PMD 0
Oops: 0000 1 SMP
last sysfs file: /sys/module/fld/initstate
CPU 0
Modules linked in: lmv(U) obdfilter(U) ost(U) cmm(U) osd_ldiskfs(U) mdt(U) mdd(U) mds(U) fsfilt_ldiskfs(U) mgs(U) mgc(U) ldiskfs(U) jbd2 lustre(U) lquota(U) lov(U) osc(U) mdc(U) fid(U) fld(U) ksocklnd(U) ptlrpc(U) obdclass(U) lnet(U) lvfs(U) sha512_generic sha256_generic crc32c_intel libcfs(U) nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc fcoe libfcoe libfc scsi_transport_fc scsi_tgt 8021q garp stp llc fuse vmhgfs(U) vsock(U) ipv6 dm_mirror dm_region_hash dm_log uinput ppdev parport_pc parport e1000 vmware_balloon sg vmci(U) i2c_piix4 i2c_core shpchp ext3 jbd mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix mptspi mptscsih mptbase scsi_transport_spi dm_mod [last unloaded: speedstep_lib]
Pid: 6821, comm: cat Tainted: G W ---------------- 2.6.32-220.17.1.el6_lustreb_neo_stable_135_2 #1 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
RIP: 0010:[<ffffffffa011c043>] [<ffffffffa011c043>] lprocfs_read_helper+0x23/0x120 [lvfs]
RSP: 0018:ffff8800065e7de8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000002 RCX: 3333333333333333
RDX: 0000000000000000 RSI: 0000000000000007 RDI: 00000000000000a0
RBP: ffff8800065e7de8 R08: 0000000000000007 R09: 00000000fffffffe
R10: 0000000000000000 R11: 0a7570632d726570 R12: 0000000000000021
R13: ffff8800396b4200 R14: 0000000000001000 R15: ffff880006480000
FS: 00007f11a0368700(0000) GS:ffff880003400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000000000a0 CR3: 00000000065bc000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process cat (pid: 6821, threadinfo ffff8800065e6000, task ffff88000c9bca80)
Stack:
ffff8800065e7e38 ffffffffa0ba2c07 ffff8800065e7e28 0000000000000000
<0> ffff8800065e7f48 ffff8800065e7f48 0000000000008000 0000000001047000
<0> ffff88000c9cbe00 ffff880006480000 ffff8800065e7e98 ffffffffa0566b33
Call Trace:
[<ffffffffa0ba2c07>] lprocfs_rd_percpu_rebuild+0x67/0x110 [mdd]
[<ffffffffa0566b33>] lprocfs_fops_read+0xf3/0x1f0 [obdclass]
[<ffffffff811db61e>] proc_reg_read+0x7e/0xc0
[<ffffffff811772e5>] vfs_read+0xb5/0x1a0
[<ffffffff810d4a12>] ? audit_syscall_entry+0x272/0x2a0
[<ffffffff81177421>] sys_read+0x51/0x90
[<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
Code: 50 50 c9 c3 90 90 90 90 55 48 89 e5 0f 1f 44 00 00 31 c0 48 85 ff 41 89 f0 0f 84 c9 00 00 00 66 0f 1f 84 00 00 00 00 00 83 fe 07 <8b> 17 77 11 42 ff 24 c5 40 fc 11 a0 90 8b 47 08 0f 1f 44 00 00
RIP [<ffffffffa011c043>] lprocfs_read_helper+0x23/0x120 [lvfs]
RSP <ffff8800065e7de8>
CR2: 00000000000000a0
crash> files
PID: 6821 TASK: ffff88000c9bca80 CPU: 0 COMMAND: "cat"
ROOT: / CWD: /proc/fs/lustre/mdd/lustre-MDT0000
FD FILE DENTRY INODE TYPE PATH
0 ffff88000247b680 ffff88003eb96b00 ffff88003ce71c18 CHR /dev/pts/2
1 ffff88000247b680 ffff88003eb96b00 ffff88003ce71c18 CHR /dev/pts/2
2 ffff88000247b680 ffff88003eb96b00 ffff88003ce71c18 CHR /dev/pts/2
3 ffff880006822440 ffff88003cde8c00 ffff88003b485a78 REG /proc/fs/lustre/mdd/lustre-MDT0000/per_cpu_rebuild
crash>

Comments

[parinay v kondekar (Inactive)]

Xyratex bug id - MRP-1071.

I am uploading the patch for review.

Thanks

[parinay v kondekar (Inactive)]

Ref - https://jira.hpdd.intel.com/browse/LU-591 patch http://review.whamcloud.com/#/c/1562/

[Oleg Drokin]

I don't see any references to per_cpu_rebuild in our codebase.
Do you have any patches applied on top of 2.1.0? Can you reproduce with a more modern Lustre version?

[parinay v kondekar (Inactive)]

Oleg,
I realized my patch is over the patch I mentioned above, which hasn't made it to landing state here. So essentially that's the reason why its not in your codebase. IMO can be closed.sorry for the trouble.

Thanks

[Peter Jones]

Thanks for letting us know Parinay!