[LU-2409] Interop 2.3<->2.4 Failure on test suite replay-single test_70b: kernel panic

Fri, 30 Nov 2012 14:41:55 +0000
This issue was created by maloo for sarah <sarah@whamcloud.com>
This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/3d209190-3981-11e2-9fda-52540035b04c.
The sub-test test_70b failed with the following error:
test failed to respond and timed out
OST console log
06:00:55:Lustre: DEBUG MARKER: == replay-single test 70b: mds recovery; 2 clients == 06:00:42 (1354024842)
06:00:55:Lustre: DEBUG MARKER: /usr/sbin/lctl mark Started rundbench load pid=10236 ...
06:00:55:Lustre: DEBUG MARKER: Started rundbench load pid=10236 ...
06:01:36:skb_over_panic: text:ffffffffa0109de1 len:650 put:650 head:(null) data:(null) tail:0x28a end:0x0 dev:<NULL>
06:01:36:BUG: scheduling while atomic: ll_ost_io00_017/11630/0x10000100
06:01:36:Modules linked in: osd_ldiskfs(U) fsfilt_ldiskfs(U) ldiskfs(U) lustre(U) ofd(U) ost(U) cmm(U) mdt(U) mdd(U) mds(U) mgs(U) jbd2 obdecho(U) mgc(U) lquota(U) lov(U) osc(U) mdc(U) lmv(U) fid(U) fld(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) sha512_generic sha256_generic libcfs(U) nfs fscache nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ipv6 ib_sa ib_mad ib_core microcode virtio_balloon 8139too 8139cp mii i2c_piix4 i2c_core ext3 jbd mbcache virtio_blk virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
06:01:36:CPU 0 
06:01:36:Modules linked in: osd_ldiskfs(U) fsfilt_ldiskfs(U) ldiskfs(U) lustre(U) ofd(U) ost(U) cmm(U) mdt(U) mdd(U) mds(U) mgs(U) jbd2 obdecho(U) mgc(U) lquota(U) lov(U) osc(U) mdc(U) lmv(U) fid(U) fld(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) sha512_generic sha256_generic libcfs(U) nfs fscache nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ipv6 ib_sa ib_mad ib_core microcode virtio_balloon 8139too 8139cp mii i2c_piix4 i2c_core ext3 jbd mbcache virtio_blk virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
06:01:36:
06:01:37:Pid: 11630, comm: ll_ost_io00_017 Not tainted 2.6.32-279.5.1.el6_lustre.gb16fe80.x86_64 #1 Red Hat KVM
06:01:37:RIP: 0010:[<ffffffff8127e7f9>]  [<ffffffff8127e7f9>] memset+0x49/0xc0
06:01:37:RSP: 0018:ffff88007553f918  EFLAGS: 00000202
06:01:37:RAX: 0000000000000000 RBX: ffff88007553f9b0 RCX: 00000000000539c8
06:01:37:RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff880071183dc0
06:01:37:RBP: ffffffff8100ba4e R08: 000000000000000a R09: 0000000000000000
06:01:38:R10: ffff880070d6b000 R11: 0000000001900000 R12: ffffea00018aef68
06:01:38:R13: ffff880072aad928 R14: ffff880072aad910 R15: ffffffff8116d011
06:01:38:FS:  00007fec10465700(0000) GS:ffff880002200000(0000) knlGS:0000000000000000
06:01:38:CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
06:01:39:CR2: 00007fb06dfe2000 CR3: 0000000037fa9000 CR4: 00000000000006f0
06:01:39:DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
06:01:39:DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
06:01:39:Process ll_ost_io00_017 (pid: 11630, threadinfo ffff88007553e000, task ffff880075534ae0)
06:01:39:Stack:
06:01:39: ffffffffa0e23e1e 0000000000000001 0000000000000001 ffff880070d6b000
06:01:39:<d> 00000000000018ff ffff880075558000 ffff880072aad7f0 ffff880075558bd0
06:01:39:<d> ffff88006fc50000 ffff88007553f9b0 ffffffffa0e22a6e 0000000050b4c7b6
06:01:39:Call Trace:
06:01:40: [<ffffffffa0e23e1e>] ? osd_write_prep+0x35e/0x420 [osd_ldiskfs]
06:01:40: [<ffffffffa0e22a6e>] ? osd_bufs_get+0x1fe/0x390 [osd_ldiskfs]
06:01:40: [<ffffffffa0c5d8d1>] ? ofd_preprw+0x7a1/0x1060 [ofd]
06:01:40: [<ffffffffa0c2381c>] ? obd_preprw+0x12c/0x3d0 [ost]
06:01:40: [<ffffffffa0c2a98a>] ? ost_brw_write+0x87a/0x1600 [ost]
06:01:40: [<ffffffff8127ce76>] ? vsnprintf+0x2b6/0x5f0
06:01:40: [<ffffffffa07751dc>] ? lustre_msg_get_version+0x8c/0x100 [ptlrpc]
06:01:40: [<ffffffffa0775338>] ? lustre_msg_check_version+0xe8/0x100 [ptlrpc]
06:01:40: [<ffffffffa0c3102c>] ? ost_handle+0x360c/0x4850 [ost]
06:01:40: [<ffffffffa04465f1>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
06:01:40: [<ffffffffa04423f4>] ? libcfs_id2str+0x74/0xb0 [libcfs]
06:01:40: [<ffffffffa0784b3c>] ? ptlrpc_server_handle_request+0x41c/0xe00 [ptlrpc]
06:01:40: [<ffffffffa043665e>] ? cfs_timer_arm+0xe/0x10 [libcfs]
06:01:40: [<ffffffffa077bf37>] ? ptlrpc_wait_event+0xa7/0x2a0 [ptlrpc]
06:01:40: [<ffffffff810533f3>] ? __wake_up+0x53/0x70
06:01:40: [<ffffffffa0786111>] ? ptlrpc_main+0xbf1/0x19e0 [ptlrpc]
06:01:40: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
06:01:40: [<ffffffff8100c14a>] ? child_rip+0xa/0x20
06:01:40: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
06:01:40: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
06:01:40: [<ffffffff8100c140>] ? child_rip+0x0/0x20
06:01:40:Code: 75 7e 44 89 d9 c1 e9 06 74 38 0f 1f 84 00 00 00 00 00 ff c9 48 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89 47 28 <48> 89 47 30 48 89 47 38 48 8d 7f 40 75 d9 66 0f 1f 84 00 00 00 
06:01:40:Call Trace:
06:01:40: [<ffffffffa0e23e1e>] ? osd_write_prep+0x35e/0x420 [osd_ldiskfs]
06:01:41: [<ffffffffa0e22a6e>] ? osd_bufs_get+0x1fe/0x390 [osd_ldiskfs]
06:01:41: [<ffffffffa0c5d8d1>] ? ofd_preprw+0x7a1/0x1060 [ofd]
06:01:41: [<ffffffffa0c2381c>] ? obd_preprw+0x12c/0x3d0 [ost]
06:01:41: [<ffffffffa0c2a98a>] ? ost_brw_write+0x87a/0x1600 [ost]
06:01:41: [<ffffffff8127ce76>] ? vsnprintf+0x2b6/0x5f0
06:01:41: [<ffffffffa07751dc>] ? lustre_msg_get_version+0x8c/0x100 [ptlrpc]
06:01:41: [<ffffffffa0775338>] ? lustre_msg_check_version+0xe8/0x100 [ptlrpc]
06:01:41: [<ffffffffa0c3102c>] ? ost_handle+0x360c/0x4850 [ost]
06:01:41: [<ffffffffa04465f1>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
06:01:41: [<ffffffffa04423f4>] ? libcfs_id2str+0x74/0xb0 [libcfs]
06:01:41: [<ffffffffa0784b3c>] ? ptlrpc_server_handle_request+0x41c/0xe00 [ptlrpc]
06:01:41: [<ffffffffa043665e>] ? cfs_timer_arm+0xe/0x10 [libcfs]
06:01:41: [<ffffffffa077bf37>] ? ptlrpc_wait_event+0xa7/0x2a0 [ptlrpc]
06:01:41: [<ffffffff810533f3>] ? __wake_up+0x53/0x70
06:01:41: [<ffffffffa0786111>] ? ptlrpc_main+0xbf1/0x19e0 [ptlrpc]
06:01:41: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
06:01:41: [<ffffffff8100c14a>] ? child_rip+0xa/0x20
06:01:41: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
06:01:41: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
06:01:41: [<ffffffff8100c140>] ? child_rip+0x0/0x20
06:01:41:------------[ cut here ]------------
06:01:41:kernel BUG at net/core/skbuff.c:127!
06:01:41:invalid opcode: 0000 [#1] SMP 
06:01:41:last sysfs file: /sys/module/ptlrpc/parameters/ldlm_enqueue_min
06:01:41:CPU 0 
06:01:41:Modules linked in: osd_ldiskfs(U) fsfilt_ldiskfs(U) ldiskfs(U) lustre(U) ofd(U) ost(U) cmm(U) mdt(U) mdd(U) mds(U) mgs(U) jbd2 obdecho(U) mgc(U) lquota(U) lov(U) osc(U) mdc(U) lmv(U) fid(U) fld(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) sha512_generic sha256_generic libcfs(U) nfs fscache nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ipv6 ib_sa ib_mad ib_core microcode virtio_balloon 8139too 8139cp mii i2c_piix4 i2c_core ext3 jbd mbcache virtio_blk virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
06:01:41:
06:01:41:Pid: 11630, comm: ll_ost_io00_017 Not tainted 2.6.32-279.5.1.el6_lustre.gb16fe80.x86_64 #1 Red Hat KVM
Whamcloud Community JIRA

[LU-2409] Interop 2.3<->2.4 Failure on test suite replay-single test_70b: kernel panic
