15:29:42:Lustre: DEBUG MARKER: == lustre-rsync-test test 2a: Replicate files created by dbench. == 15:29:41 (1354058981)
15:29:54:skb_over_panic: text:ffffffffa0109de1 len:650 put:650 head:(null) data:(null) tail:0x28a end:0x0 dev:<NULL>
15:29:54:BUG: scheduling while atomic: ll_ost00_003/7643/0x10000100
15:29:54:Modules linked in: osd_ldiskfs(U) fsfilt_ldiskfs(U) ldiskfs(U) lustre(U) ofd(U) ost(U) cmm(U) mdt(U) mdd(U) mds(U) mgs(U) jbd2 obdecho(U) mgc(U) lquota(U) lov(U) osc(U) mdc(U) lmv(U) fid(U) fld(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) sha512_generic sha256_generic libcfs(U) nfs fscache nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ipv6 ib_sa ib_mad ib_core microcode virtio_balloon 8139too 8139cp mii i2c_piix4 i2c_core ext3 jbd mbcache virtio_blk virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
15:29:54:CPU 0
15:29:54:Modules linked in: osd_ldiskfs(U) fsfilt_ldiskfs(U) ldiskfs(U) lustre(U) ofd(U) ost(U) cmm(U) mdt(U) mdd(U) mds(U) mgs(U) jbd2 obdecho(U) mgc(U) lquota(U) lov(U) osc(U) mdc(U) lmv(U) fid(U) fld(U) ptlrpc(U) obdclass(U) lvfs(U) ksocklnd(U) lnet(U) sha512_generic sha256_generic libcfs(U) nfs fscache nfsd lockd nfs_acl auth_rpcgss exportfs autofs4 sunrpc ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_addr ipv6 ib_sa ib_mad ib_core microcode virtio_balloon 8139too 8139cp mii i2c_piix4 i2c_core ext3 jbd mbcache virtio_blk virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]
15:29:54:
15:29:54:Pid: 7643, comm: ll_ost00_003 Not tainted 2.6.32-279.5.1.el6_lustre.gb16fe80.x86_64 #1 Red Hat KVM
15:29:54:RIP: 0010:[<ffffffff8127cca1>] [<ffffffff8127cca1>] vsnprintf+0xe1/0x5f0
15:29:54:RSP: 0018:ffff88007694d6c0 EFLAGS: 00000282
15:29:54:RAX: 0000000000000025 RBX: ffff88007694d750 RCX: 0000000000000007
15:29:54:RDX: 0000000000000007 RSI: ffffffffa07d552f RDI: ffff88007869df67
15:29:54:RBP: ffffffff8100ba4e R08: 00000000203a6b63 R09: 646461206f742074
15:29:54:R10: 6968742064646120 R11: 0a3a6b636f6c2073 R12: 3939623700000001
15:29:55:R13: 0000000000000020 R14: 0000006cffffffff R15: 0000000000000282
15:29:55:FS: 00007f0722de8700(0000) GS:ffff880002200000(0000) knlGS:0000000000000000
15:29:55:CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
15:29:55:CR2: 00007f0ffa124000 CR3: 000000007b02c000 CR4: 00000000000006f0
15:29:55:DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
15:29:55:DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
15:29:55:Process ll_ost00_003 (pid: 7643, threadinfo ffff88007694c000, task ffff88007696d500)
15:29:55:Stack:
15:29:55: 0000000000000004 0000000affffffff ffffffffffffffff 0000000000000000
15:29:55:<d> 0000000000000007 7869df4b8109caaa 00000000000000b5 ffff88007869df4b
15:29:55:<d> 0000000000000000 0000000affffffff ffffffffffffffff ffffffffa043527b
15:29:55:Call Trace:
15:29:55: [<ffffffffa043527b>] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs]
15:29:55: [<ffffffffa0445d60>] ? libcfs_debug_vmsg2+0x310/0xb60 [libcfs]
15:29:55: [<ffffffffa0442375>] ? libcfs_nid2str+0x155/0x160 [libcfs]
15:29:55: [<ffffffffa0728b23>] ? _ldlm_lock_debug+0x513/0x5d0 [ptlrpc]
15:29:55: [<ffffffffa07313f8>] ? ldlm_resource_add_lock+0xf8/0x190 [ptlrpc]
15:29:55: [<ffffffffa07439e7>] ? ldlm_extent_add_lock+0x107/0x370 [ptlrpc]
15:29:55: [<ffffffffa072c2d3>] ? ldlm_grant_lock+0x4a3/0x6f0 [ptlrpc]
15:29:55: [<ffffffffa0744e20>] ? ldlm_process_extent_lock+0x380/0xaf0 [ptlrpc]
15:29:55: [<ffffffffa072c925>] ? ldlm_lock_enqueue+0x405/0x8f0 [ptlrpc]
15:29:55: [<ffffffffa07549bf>] ? ldlm_handle_enqueue0+0x48f/0xf70 [ptlrpc]
15:29:55: [<ffffffffa0755506>] ? ldlm_handle_enqueue+0x66/0x70 [ptlrpc]
15:29:55: [<ffffffffa0755510>] ? ldlm_server_completion_ast+0x0/0x610 [ptlrpc]
15:29:55: [<ffffffffa0c26e20>] ? ost_blocking_ast+0x0/0xe60 [ost]
15:29:55: [<ffffffffa074f600>] ? ldlm_server_glimpse_ast+0x0/0x370 [ptlrpc]
15:29:55: [<ffffffffa0c2f7a3>] ? ost_handle+0x1d83/0x4850 [ost]
15:29:55: [<ffffffffa04465f1>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
15:29:55: [<ffffffffa04423f4>] ? libcfs_id2str+0x74/0xb0 [libcfs]
15:29:55: [<ffffffffa0784b3c>] ? ptlrpc_server_handle_request+0x41c/0xe00 [ptlrpc]
15:29:55: [<ffffffffa043665e>] ? cfs_timer_arm+0xe/0x10 [libcfs]
15:29:55: [<ffffffffa077bf37>] ? ptlrpc_wait_event+0xa7/0x2a0 [ptlrpc]
15:29:55: [<ffffffff810533f3>] ? __wake_up+0x53/0x70
15:29:55: [<ffffffffa0786111>] ? ptlrpc_main+0xbf1/0x19e0 [ptlrpc]
15:29:55: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
15:29:55: [<ffffffff8100c14a>] ? child_rip+0xa/0x20
15:29:55: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
15:29:55: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
15:29:55: [<ffffffff8100c140>] ? child_rip+0x0/0x20
15:29:55:Code: 56 08 8b 10 e9 a0 03 00 00 90 89 c2 ff 24 d5 a0 35 63 81 0f 1f 80 00 00 00 00 4c 39 fb 73 03 c6 03 25 41 0f b6 04 24 48 83 c3 01 <84> c0 4d 89 e5 75 90 48 83 7d a0 00 74 0c 49 39 df 0f 86 40 03
15:29:55:Call Trace:
15:29:55: [<ffffffffa043527b>] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs]
15:29:55: [<ffffffffa0445d60>] ? libcfs_debug_vmsg2+0x310/0xb60 [libcfs]
15:29:55: [<ffffffffa0442375>] ? libcfs_nid2str+0x155/0x160 [libcfs]
15:29:55: [<ffffffffa0728b23>] ? _ldlm_lock_debug+0x513/0x5d0 [ptlrpc]
15:29:55: [<ffffffffa07313f8>] ? ldlm_resource_add_lock+0xf8/0x190 [ptlrpc]
15:29:55: [<ffffffffa07439e7>] ? ldlm_extent_add_lock+0x107/0x370 [ptlrpc]
15:29:55: [<ffffffffa072c2d3>] ? ldlm_grant_lock+0x4a3/0x6f0 [ptlrpc]
15:29:55: [<ffffffffa0744e20>] ? ldlm_process_extent_lock+0x380/0xaf0 [ptlrpc]
15:29:55: [<ffffffffa072c925>] ? ldlm_lock_enqueue+0x405/0x8f0 [ptlrpc]
15:29:55: [<ffffffffa07549bf>] ? ldlm_handle_enqueue0+0x48f/0xf70 [ptlrpc]
15:29:55: [<ffffffffa0755506>] ? ldlm_handle_enqueue+0x66/0x70 [ptlrpc]
15:29:55: [<ffffffffa0755510>] ? ldlm_server_completion_ast+0x0/0x610 [ptlrpc]
15:29:55: [<ffffffffa0c26e20>] ? ost_blocking_ast+0x0/0xe60 [ost]
15:29:55: [<ffffffffa074f600>] ? ldlm_server_glimpse_ast+0x0/0x370 [ptlrpc]
15:29:55: [<ffffffffa0c2f7a3>] ? ost_handle+0x1d83/0x4850 [ost]
15:29:55: [<ffffffffa04465f1>] ? libcfs_debug_msg+0x41/0x50 [libcfs]
15:29:55: [<ffffffffa04423f4>] ? libcfs_id2str+0x74/0xb0 [libcfs]
15:29:55: [<ffffffffa0784b3c>] ? ptlrpc_server_handle_request+0x41c/0xe00 [ptlrpc]
15:29:55: [<ffffffffa043665e>] ? cfs_timer_arm+0xe/0x10 [libcfs]
15:29:55: [<ffffffffa077bf37>] ? ptlrpc_wait_event+0xa7/0x2a0 [ptlrpc]
15:29:55: [<ffffffff810533f3>] ? __wake_up+0x53/0x70
15:29:55: [<ffffffffa0786111>] ? ptlrpc_main+0xbf1/0x19e0 [ptlrpc]
15:29:55: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
15:29:55: [<ffffffff8100c14a>] ? child_rip+0xa/0x20
15:29:55: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
15:29:55: [<ffffffffa0785520>] ? ptlrpc_main+0x0/0x19e0 [ptlrpc]
15:29:55: [<ffffffff8100c140>] ? child_rip+0x0/0x20
15:29:55:BUG: unable to handle kernel NULL pointer dereference at (null)