In a Multi-Rail setup using ethernet interfaces, it appears like there is a wrong mapping between the LNet level NID and the ethernet interfaces.
When sending traffic, LNet reports all NIDs are being used for traffic, but when we use netstat -i to monitor LNet traffic, we only see traffic on a subset of the interfaces.

When we restrict traffic from LNet on a subset of the NIDs, even for that subset, the interfaces don't match. For example netstat -i can show traffic on eth0 and eth2. But LNet shows that it's using eth1 and eth2.

However, when using iperf, all ethernet interfaces are used according to netstat -i

This behavior is easily reproducible on a simple 2 VM MR setup.