[LU-13625] socklnd can select the wrong IP address to bind socket to

Related — Sun, 9 May 2021 15:02:34 +0000

In choose_ipv4_src() it doesn't matter which IP address you bind to as long as you bind to the interface. The reasoning is that the only important thing is you bind to the interface to make sure traffic goes out the NIC you expect.
However, I see a drawback with not binding to an IP address. I'll mention two tests I did to show the drawback. The first one is on the nose test:
TEST ONE
1. I configure 2 IP addresses on one interface.
2. I add a REJECT rule on the second IP address. such that all traffic using the second IP address is dropped
3. I configure LNet to use that interface.
4. All LNet traffic binds against the second address and all traffic is dropped
TEST TWO
1. I configure routing and rules such that anything using the first IP address on the interfaces egress the desired NIC
2. Run an lnet self test and monitor traffic via "netstat -i"
3. Since the socket is being bound to the second IP on the interfaces the rules are not being hit and traffic is not distributed as expected

From the below stats, only eth0 is being using for TXs

Kernel Interface table
Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0      1500   115139      0   1840 0       1025644      0      0      0 BMRU
eth1      1500   115627      0   3597 0            24      0      0      0 BMRU
eth2      1500   179972      0    132 0            23      0      0      0 BMRU
lo       65536      372      0      0 0           372      0      0      0 LRU

LNet is a virtual networking layer on top of the physical network. The admin should have control over not only the interface LNet uses, but also the IP address. I admit, that currently our user space tools only bind to the interface (although ip2nets allows specifying IP ranges). However, I see situations, especially as ethernet becomes more widely used on sites, that admins might want the control to configure multiple IP addresses, route packets using each IP address differently and have Lustre go over one of the specific routes, while dealing with applications using the second IP address differently.

Whamcloud Community JIRA

[LU-13625] socklnd can select the wrong IP address to bind socket to