2011-06-23 14:54:31 LustreError: 9730:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 121636742: cookie 0x0 req@ffff812001bdf000 x1372347904386655/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866077 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:54:31 LustreError: 9725:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 80353597: cookie 0x0 req@ffff811f08c9e000 x1372347904386656/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866088 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:55:21 LustreError: 9696:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 80447640: cookie 0x0 req@ffff812f3ed22c00 x1372347904386678/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866138 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:55:22 LustreError: 9705:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 81301061: cookie 0x0 req@ffff813026526800 x1372347904386949/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866139 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:55:22 LustreError: 9705:0:(mds_open.c:1693:mds_close()) Skipped 270 previous similar messages 2011-06-23 14:55:24 LustreError: 9718:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 81366046: cookie 0x0 req@ffff813fd3744000 x1372347904387510/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866141 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:55:24 LustreError: 9718:0:(mds_open.c:1693:mds_close()) Skipped 560 previous similar messages 2011-06-23 14:55:28 LustreError: 9720:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 81399082: cookie 0x0 req@ffff813fd2860400 x1372347904388665/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866145 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:55:28 LustreError: 9720:0:(mds_open.c:1693:mds_close()) Skipped 1154 previous similar messages 2011-06-23 14:55:36 LustreError: 9744:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 80481620: cookie 0x0 req@ffff813025739c50 x1372347904390273/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866153 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:55:36 LustreError: 9744:0:(mds_open.c:1693:mds_close()) Skipped 1607 previous similar messages 2011-06-23 14:55:52 ----------- [cut here ] --------- [please bite here ] --------- 2011-06-23 14:55:52 Kernel BUG at mm/slab.c:3482 2011-06-23 14:55:52 invalid opcode: 0000 [1] SMP 2011-06-23 14:55:52 last sysfs file: /devices/pci0000:80/0000:80:00.0/irq 2011-06-23 14:55:52 CPU 7 2011-06-23 14:55:52 Modules linked in: mds(U) fsfilt_ldiskfs(U) mgs(U) mgc(U) ldiskfs(U) jbd2(U) crc16(U) lustre(U) lov(U) mdc(U) lquota(U) osc(U) ko2iblnd(U) ptlrpc(U) obdclass(U) lnet(U) lvfs(U) libcfs(U) ib_srp(U) ib_ipoib(U) rdma_ucm(U) rdma_cm(U) iw_cm(U) ib_addr(U) ib_ucm(U) ib_uverbs(U) ib_umad(U) mlx4_ib(U) mlx4_core(U) ipoib_helper(U) ib_cm(U) ib_sa(U) ib_mad(U) ib_core(U) ipv6(U) xfrm_nalgo(U) crypto_api(U) dm_mirror(U) dm_log(U) dm_multipath(U) scsi_dh(U) dm_mod(U) raid10(U) video(U) backlight(U) sbs(U) power_meter(U) i2c_ec(U) dell_wmi(U) wmi(U) button(U) battery(U) asus_acpi(U) acpi_memhotplug(U) ac(U) parport_pc(U) lp(U) parport(U) sd_mod(U) sg(U) floppy(U) mptsas(U) sata_nv(U) mptscsih(U) pcspkr(U) libata(U) amd64_edac_mod(U) i2c_nforce2(U) mptbase(U) ohci_hcd(U) edac_mc(U) i2c_core(U) scsi_transport_sas(U) tpm_tis(U) k10temp(U) tpm(U) shpchp(U) scsi_mod(U) hwmon(U) tpm_bios(U) ide_cd(U) serio_raw(U) cdrom(U) nfs(U) nfs_acl(U) lockd(U) fscache(U) sunrpc(U) e1000(U) 2011-06-23 14:55:52 Pid: 9565, comm: ldlm_cn_02 Tainted: G 2.6.18-238.12.1.el5_lustre.g266a955 #1 2011-06-23 14:55:52 RIP: 0010:[] [] kmem_cache_free+0x89/0x1dd 2011-06-23 14:55:52 RSP: 0018:ffff814026f15c80 EFLAGS: 00010207 2011-06-23 14:55:52 RAX: 0000000000000080 RBX: ffff81100104a840 RCX: 0000000000000200 2011-06-23 14:55:52 RDX: ffff810138039030 RSI: 0000000038039030 RDI: 0000000000000001 2011-06-23 14:55:52 RBP: ffff81102a4a9a00 R08: 0000000000000000 R09: 0000000000000000 2011-06-23 14:55:52 R10: ffff81100104a840 R11: 0000000000000158 R12: ffff811028ee5040 2011-06-23 14:55:52 R13: ffff81100104a840 R14: ffff81302530a0c0 R15: 00000000000000f6 2011-06-23 14:55:52 FS: 00002aaaab01f6e0(0000) GS:ffff811038155e40(0000) knlGS:0000000000000000 2011-06-23 14:55:52 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b 2011-06-23 14:55:52 CR2: 00002aaaaad626e0 CR3: 0000002f43886000 CR4: 00000000000006e0 2011-06-23 14:55:52 Process ldlm_cn_02 (pid: 9565, threadinfo ffff814026f14000, task ffff814037bab040) 2011-06-23 14:55:52 Stack: ffff81100104a898 ffff81100104a840 ffff81100104a840 ffff81102a4a9a00 2011-06-23 14:55:52 0000000000000000 ffff81100104a840 ffff81302530a0c0 ffffffff886ca3eb 2011-06-23 14:55:52 ffff810fa10c9000 ffffffff886c58d2 ffff814026f15cd0 ffff814026f15cd0 2011-06-23 14:55:52 Call Trace: 2011-06-23 14:55:52 [] :ptlrpc:ldlm_resource_putref_internal+0x3ab/0x460 2011-06-23 14:55:52 LustreError: 3297:0:(mds_open.c:1693:mds_close()) @@@ no handle for file close ino 81431268: cookie 0x0 req@ffff813fd3b6b000 x1372347904393553/t0 o35->c339861d-0c0a-b9ee-a39d-9cfe591452c1@NET_0x50000c0a87223_UUID:0/0 lens 408/864 e 0 to 0 dl 1308866169 ref 1 fl Interpret:/0/0 rc 0/0 2011-06-23 14:55:52 LustreError: 3297:0:(mds_open.c:1693:mds_close()) Skipped 3269 previous similar messages 2011-06-23 14:55:52 [] :ptlrpc:ldlm_lock_put+0x372/0x3d0 2011-06-23 14:55:52 [] :ptlrpc:ldlm_request_cancel+0x222/0x330 2011-06-23 14:55:52 [] :ptlrpc:lustre_pack_reply+0x29/0xb0 2011-06-23 14:55:52 [] :ptlrpc:ldlm_handle_cancel+0x21d/0x2e0 2011-06-23 14:55:52 [] :ptlrpc:lustre_msg_get_opc+0x35/0xf0 2011-06-23 14:55:52 [] :ptlrpc:ldlm_cancel_handler+0x308/0x690 2011-06-23 14:55:52 [] :ptlrpc:lustre_msg_get_opc+0x35/0xf0 2011-06-23 14:55:52 [] :ptlrpc:ptlrpc_server_handle_request+0x989/0xe00 2011-06-23 14:55:52 [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 2011-06-23 14:55:52 [] __wake_up_common+0x3e/0x68 2011-06-23 14:55:52 [] :ptlrpc:ptlrpc_main+0xf66/0x1120 2011-06-23 14:55:52 [] child_rip+0xa/0x11 2011-06-23 14:55:52 [] :ptlrpc:ptlrpc_main+0x0/0x1120 2011-06-23 14:55:52 [] child_rip+0x0/0x11 2011-06-23 14:55:52 2011-06-23 14:55:52 2011-06-23 14:55:52 Code: 0f 0b 68 d8 a0 2b 80 c2 9a 0d 9c 41 5e fa 65 8b 04 25 2c 00 2011-06-23 14:55:52 RIP [] kmem_cache_free+0x89/0x1dd 2011-06-23 14:55:52 RSP 2011-06-23 14:55:52 REWRITING MCP55 CFG REG 2011-06-23 14:55:52 CFG = c1 2011-06-23 14:55:52 Linux version 2.6.18-238.12.1.el5_lustre.g266a955 (jenkins@rhel5-64-build.lab.whamcloud.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) #1 SMP Fri Jun 10 16:39:27 PDT 2011