Aug 31 12:31:23 elm-rcf-io1-s1 kernel: libcfs: loading out-of-tree module taints kernel.
Aug 31 12:31:23 elm-rcf-io1-s1 kernel: libcfs: module verification failed: signature and/or required key missing - tainting kernel
Aug 31 12:31:23 elm-rcf-io1-s1 kernel: libcfs: HW NUMA nodes: 2, HW CPU cores: 64, npartitions: 2
Aug 31 12:31:23 elm-rcf-io1-s1 kernel: alg: No test for adler32 (adler32-zlib)
Aug 31 12:31:24 elm-rcf-io1-s1 kernel: Key type ._llcrypt registered
Aug 31 12:31:24 elm-rcf-io1-s1 kernel: Key type .llcrypt registered
Aug 31 12:31:24 elm-rcf-io1-s1 kernel: LNet: Using FastReg for registration
Aug 31 12:31:24 elm-rcf-io1-s1 kernel: LNet: Added LNI 10.4.0.21@o2ib [8/256/0/180]
Aug 31 12:31:49 elm-rcf-io1-s1 kernel: LNet: devevent: status=10, iface=lo ifindex 1 state 0
Aug 31 12:31:49 elm-rcf-io1-s1 kernel: LNet: devevent: status=2, iface=lo ifindex 1 state 0
Aug 31 12:31:49 elm-rcf-io1-s1 kernel: LNet: addrevent: status 2 ip addr 127.0.0.1, netmask 255.0.0.0.
Aug 31 12:31:49 elm-rcf-io1-s1 kernel: LNet: devevent: status=6, iface=lo ifindex 1 state 0
Aug 31 12:32:14 elm-rcf-io1-s1 kernel: LNet: devevent: status=17, iface=lo ifindex 1 state 0
Aug 31 12:32:14 elm-rcf-io1-s1 kernel: LNet: devevent: status=5, iface=lo ifindex 1 state 0
Aug 31 12:32:14 elm-rcf-io1-s1 kernel: LNet: addrevent: status 1 ip addr 127.0.0.1, netmask 255.0.0.0.
Aug 31 12:32:14 elm-rcf-io1-s1 kernel: LNet: devevent: status=14, iface=lo ifindex 1 state 0
Aug 31 12:32:14 elm-rcf-io1-s1 kernel: LNet: devevent: status=1, iface=lo ifindex 1 state 0
Aug 31 12:32:19 elm-rcf-io1-s1 kernel: LNet: alloc '(nlist)': 8 at 00000000f304d176.
Aug 31 12:32:19 elm-rcf-io1-s1 kernel: LNet: kfreed 'nlist': 8 at 00000000f304d176.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(plist)': 64 at 000000009240c72a.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(pbuf)': 3225 at 000000008e494a7c.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: slab-alloced 'lmd': 136 at 00000000c02d2b45.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc 'rspt': 56 at 00000000da09a9e5.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: LNetGet -> 12345-10.4.0.21@o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(lp)': 312 at 00000000f75cab87.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: 00000000f75cab87 nid 10.4.0.21@o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(lpn)': 72 at 00000000716ab038.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: 00000000716ab038 net o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(lpni)': 336 at 000000002c3e81d2.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: 000000002c3e81d2 nid 10.4.0.21@o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: peer 10.4.0.21@o2ib NID 10.4.0.21@o2ib flags 0x0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: peer 10.4.0.21@o2ib: 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Source ANY to NMR:  10.4.0.21@o2ib local destination
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: selected best_ni 10.4.0.21@o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Setting preferred local NID 10.4.0.21@o2ib on NMR peer 10.4.0.21@o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: peer 10.4.0.21@o2ib nid 10.4.0.21@o2ib: 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: 10.4.0.21@o2ib NI seq info: [1:1:256:4294967295] 10.4.0.21@o2ib LPNI seq info [1:1:8:4294967295]
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rspt_next_hop_nid = 10.4.0.21@o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: TRACE: 10.4.0.21@o2ib(10.4.0.21@o2ib:<?>) -> 10.4.0.21@o2ib(10.4.0.21@o2ib:10.4.0.21@o2ib) <?> : GET try# 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: sending 0 bytes in 0 frags to 12345-10.4.0.21@o2ib
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(peer_ni)': 120 at 00000000e189c48e.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: peer_ni[00000000e189c48e] -> 10.4.0.21@o2ib (1)++
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: peer_ni[00000000e189c48e] -> 10.4.0.21@o2ib (2)++
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: bound to port 1023
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: 10.4.0.21@o2ib Addr resolved: 0 cm_id 00000000a6c84875
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: peer_ni[00000000e189c48e] -> 10.4.0.21@o2ib (3)--
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: 10.4.0.21@o2ib: connection bound to eno12399np0:10.4.0.21:bnxt_re0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: 10.4.0.21@o2ib Route resolved: 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(conn)': 304 at 000000007050824c.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(conn->ibc_connvars)': 136 at 00000000132f9231.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(conn->ibc_rxs)': 1872 at 0000000021ce0077.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: alloc '(p)': 152 at 000000003d7f9809.
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 0: 00000000c273ed90 0xf457f000(0x185374000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 1: 0000000097fa4b5d 0xf457e000(0x167bf0000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 2: 00000000cc7275e2 0xf457d000(0x18a266000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 3: 00000000cfa048a8 0xf457c000(0x1116d7000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 4: 00000000fc7ade1f 0xf457b000(0x110c38000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 5: 000000009f77573b 0xf457a000(0x1044a4000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 6: 00000000faec7561 0xf4579000(0x16ad6a000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 7: 00000000a1ace9ec 0xf4578000(0x15494d000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 8: 00000000b2bd31d8 0xf4577000(0x18d9f3000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 9: 00000000c6792c2d 0xf4576000(0x14e626000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 10: 00000000c73a0668 0xf4575000(0x16bce9000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 11: 000000007bd33772 0xf4574000(0x165227000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 12: 00000000b642a2c0 0xf4573000(0x18a175000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 13: 0000000013cbebd8 0xf4572000(0x165ff1000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 14: 00000000efb85f17 0xf4571000(0x11c64d000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 15: 00000000a0edbd1a 0xf4570000(0x1877b5000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 16: 00000000037a8ea2 0xf456f000(0x18a1f3000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: rx 17: 00000000dffdc6cf 0xf456e000(0x14d51e000)
Aug 31 12:32:23 elm-rcf-io1-s1 kernel: LNet: Connected to 10.4.0.21@o2ib: cm_id 00000000a6c84875
Aug 31 12:32:44 elm-rcf-io1-s1 kernel: LNet: devevent: status=10, iface=lo ifindex 1 state 0
Aug 31 12:32:44 elm-rcf-io1-s1 kernel: LNet: devevent: status=2, iface=lo ifindex 1 state 0
Aug 31 12:32:44 elm-rcf-io1-s1 kernel: LNet: addrevent: status 2 ip addr 127.0.0.1, netmask 255.0.0.0.
Aug 31 12:32:44 elm-rcf-io1-s1 kernel: LNet: devevent: status=6, iface=lo ifindex 1 state 0
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: 9977:0:(o2iblnd_cb.c:3501:kiblnd_check_conns()) Timed out tx for 10.4.0.21@o2ib: 0 seconds
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: health check = 1, status = -110, hstatus = 10
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: health check: 10.4.0.21@o2ib->10.4.0.21@o2ib: GET: NETWORK_TIMEOUT
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: ping event (5 -110)
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: Process entered
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: Process leaving
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: Unlinking md 00000000c02d2b45
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: slab-freed 'md': 136 at 00000000c02d2b45.
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: ping event (6 0) unlinked
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: kfreed 'pbuf': 3225 at 000000008e494a7c.
Aug 31 12:33:12 elm-rcf-io1-s1 kernel: LNet: kfreed 'plist': 64 at 000000009240c72a.
Aug 31 12:33:54 elm-rcf-io1-s1 kernel: LNet: free 'rspt': 56 at 00000000da09a9e5.
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: 10.4.0.21@o2ib: UNREACHABLE -110 cm_id 00000000a6c84875 conn 000000007050824c
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: 10.4.0.21@o2ib: active(1), version(12), status(-100)
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: kfreed 'conn->ibc_connvars': 136 at 00000000132f9231.
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: peer_ni[00000000e189c48e] -> 10.4.0.21@o2ib (2)--
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: 10.4.0.21@o2ib notifying 10.4.0.21@o2ib: down
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: Match nid 10.4.0.21@o2ib to cpt 0
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: conn 000000007050824c state 1 nposted 0/0 c/o/r 0/0/0 ce 0 : cm_id 00000000a6c84875 qp_num 0x82 device_name bnxt_re0
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: abort connection with 10.4.0.21@o2ib
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: kfreed 'p': 152 at 000000003d7f9809.
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: kfreed 'conn->ibc_rxs': 1872 at 0000000021ce0077.
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: peer_ni[00000000e189c48e] -> 10.4.0.21@o2ib (1)--
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: kfreed 'peer_ni': 120 at 00000000e189c48e.
Aug 31 12:34:09 elm-rcf-io1-s1 kernel: LNet: kfreed 'conn': 304 at 000000007050824c.
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process entered
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process entered
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: alloc '(*hdr_pp)': 988 at 00000000cf187e20.
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process leaving (rc=0 : 0 : 0)
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: lnet ioctl cmd 3233310049
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: refs 1
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: kfreed 'hdr': 988 at 00000000cf187e20.
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process leaving (rc=0 : 0 : 0)
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process entered
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process entered
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: alloc '(*hdr_pp)': 988 at 00000000cf187e20.
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process leaving (rc=0 : 0 : 0)
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: lnet ioctl cmd 3233310049
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: refs 1
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: kfreed 'hdr': 988 at 00000000cf187e20.
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process leaving (rc=0 : 0 : 0)
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process entered
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process entered
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: alloc '(*hdr_pp)': 988 at 00000000cf187e20.
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process leaving (rc=0 : 0 : 0)
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: lnet ioctl cmd 3233310049
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: refs 1
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: kfreed 'hdr': 988 at 00000000cf187e20.
Aug 31 12:34:33 elm-rcf-io1-s1 kernel: LNet: Process leaving (rc=18446744073709551614 : -2 : fffffffffffffffe)