For Multi-Rail setups there are a few parameters which need to be set properly to allow for proper ARP resolution. It'll be best to set these parameters from LNet to avoid having the user run into unexpected behavior

net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.ib0.accept_local=1
net.ipv4.conf.ib0.arp_ignore=1
net.ipv4.conf.ib0.arp_filter=0
net.ipv4.conf.ib0.arp_announce=2
net.ipv4.conf.ib0.rp_filter=0