Details
-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
3
-
9223372036854775807
Description
[ 8125.253125] LNet: Using FastReg for registration [ 8125.831072] LNet: Added LNI 172.18.70.140@o2ib1 [128/2048/0/180] [ 8125.840305] LNet: 8866:0:(o2iblnd.c:943:kiblnd_create_conn()) peer 10.14.252.1@o2ib1 - queue depth reduced from 128 to 21 to allow for qp creation [ 8125.856172] LNetError: 4:0:(o2iblnd_cb.c:2840:kiblnd_rejected()) 10.14.252.1@o2ib1 rejected: consumer defined fatal error [ 8125.870235] LNet: 4:0:(o2iblnd_cb.c:2207:kiblnd_peer_connect_failed()) Deleting messages for 10.14.252.1@o2ib1: connection failed [ 8125.870273] LNet: 9852:0:(o2iblnd_cb.c:497:kiblnd_rx_complete()) Rx from 10.14.252.1@o2ib1 failed: 5 [ 8125.871400] BUG: unable to handle kernel paging request at ffffeb04004b9a40 [ 8125.871405] IP: [<ffffffff811dee05>] kfree+0x55/0x140 [ 8125.871406] PGD 0 [ 8125.871407] Oops: 0000 [#1] SMP [ 8125.871432] Modules linked in: ko2iblnd(OE) lnet(OE) libcfs(OE) rpcrdma ib_isert iscsi_target_mod target_core_mod ib_iser libiscsi scsi_transport_iscsi ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx5_ib edac_core intel_powerclamp coretemp intel_rapl iosf_mbi ttm iTCO_wdt drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops iTCO_vendor_support drm kvm irqbypass joydev crc32_pclmul ghash_clmulni_intel ipmi_si aesni_intel ipmi_devintf pcspkr lrw ipmi_msghandler gf128mul wmi acpi_pad glue_helper ablk_helper cryptd ioatdma acpi_power_meter sg i2c_i801 mei_me mei lpc_ich shpchp acpi_cpufreq nfsd binfmt_misc auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic hfi1(OE) rdmavt(OE) mlx5_core igb ib_core ahci devlink crct10dif_pclmul [ 8125.871436] libahci crct10dif_common ptp crc32c_intel i2c_algo_bit libata dca i2c_core pps_core dm_mirror dm_region_hash dm_log dm_mod [ 8125.871438] CPU: 17 PID: 9844 Comm: kiblnd_connd Tainted: G IOE ------------ 3.10.0-693.el7.x86_64 #1 [ 8125.871439] Hardware name: Intel Corporation S2600KPF/S2600KPF, BIOS SE5C610.86B.01.01.0008.021120151325 02/11/2015 [ 8125.871440] task: ffff880174259fa0 ti: ffff88084df30000 task.ti: ffff88084df30000 [ 8125.871442] RIP: 0010:[<ffffffff811dee05>] [<ffffffff811dee05>] kfree+0x55/0x140 [ 8125.871442] RSP: 0018:ffff88084df33dc8 EFLAGS: 00010282 [ 8125.871443] RAX: ffffeb04004b9a40 RBX: ffffc90012e69000 RCX: 0000000180080007 [ 8125.871443] RDX: 000077ff80000000 RSI: ffffea0041778400 RDI: ffffc90012e69000 [ 8125.871444] RBP: ffff88084df33de0 R08: ffff88105de10000 R09: 0000000180080007 [ 8125.871444] R10: 0000000000000001 R11: ffffea0041778400 R12: ffff8808f3fca800 [ 8125.871444] R13: ffffffffc0817294 R14: 00000000000011e0 R15: ffff881053ac3000 [ 8125.871445] FS: 0000000000000000(0000) GS:ffff88085ee40000(0000) knlGS:0000000000000000 [ 8125.871446] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 8125.871446] CR2: ffffeb04004b9a40 CR3: 00000000019f2000 CR4: 00000000001407e0 [ 8125.871447] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 8125.871447] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 8125.871447] Stack: [ 8125.871448] ffff881053ac3000 ffff8808f3fca800 ffff8808426e6000 ffff88084df33e18 [ 8125.871449] ffffffffc0817294 ffff881053ac3018 0000000000000246 00000000000003e8 [ 8125.871450] 000000005b67fd54 ffff881053ac3000 ffff88084df33ec0 ffffffffc082a5e7 [ 8125.871450] Call Trace: [ 8125.871457] [<ffffffffc0817294>] kiblnd_destroy_conn+0x184/0x6e0 [ko2iblnd] [ 8125.871462] [<ffffffffc082a5e7>] kiblnd_connd+0x6a7/0xa00 [ko2iblnd] [ 8125.871465] [<ffffffff810c4810>] ? wake_up_state+0x20/0x20 [ 8125.871468] [<ffffffffc0829f40>] ? kiblnd_check_conns+0x860/0x860 [ko2iblnd] [ 8125.871470] [<ffffffff810b098f>] kthread+0xcf/0xe0 [ 8125.871471] [<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40 [ 8125.871474] [<ffffffff816b4f18>] ret_from_fork+0x58/0x90 [ 8125.871475] [<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40 [ 8125.871484] Code: 8b 15 30 a2 81 00 48 01 d8 72 0e 48 c7 c2 00 00 00 80 48 2b 15 05 4e 84 00 48 01 d0 48 c1 e8 0c 48 c1 e0 06 48 03 05 e3 4d 84 00 <48> 8b 10 80 e6 80 0f 85 c5 00 00 00 49 89 c3 49 8b 03 a8 80 0f [ 8125.871486] RIP [<ffffffff811dee05>] kfree+0x55/0x140 [ 8125.871486] RSP <ffff88084df33dc8> [ 8125.871486] CR2: ffffeb04004b9a40
00000800:00000010:3.0:1533566495.528345:0:5501:0:(o2iblnd.c:861:kiblnd_create_conn()) alloc '(conn->ibc_rxs)': 26832 at ffffc90012e69000 (tot 75387476). 00000800:00000010:17.0:1533566495.592663:0:5553:0:(o2iblnd.c:1045:kiblnd_destroy_conn()) kfreed 'conn->ibc_rxs': 4576 at ffffc90012e69000 (tot 75470220).
Size was changed by:
00000800:00000400:17.0:1533566495.530176:0:5501:0:(o2iblnd.c:943:kiblnd_create_conn()) peer 172.18.2.3@o2ib - queue depth reduced from 128 to 21 to allow for qp creation
it shouldn't matter but LIBCFS_FREE decides whether to use kfree or vfree depending on a size.