[LU-6130] Number of LNET NI's limited

• Type: Bug
• Resolution: Unresolved
• Priority: Major
• Affects Version/s: Lustre 2.7.0
• Severity: 3
• Assignee: Serguei Smirnov
• Reporter: Blake Caldwell

Description

While investigating the number of maximum possible NI's that can be added with DLC, I found that the structure lnet_ping_info_t is only expected to have a maximum of 16 entries in pi_ni.

./lnet/include/lnet/lib-types.h:460

typedef struct {
__u32 pi_magic;
__u32 pi_features;
lnet_pid_t pi_pid;
__u32 pi_nnis;
lnet_ni_status_t pi_ni[0];
} WIRE_ATTR lnet_ping_info_t;

/* router checker data, per router */
#define LNET_MAX_RTR_NIS 16
#define LNET_PINGINFO_SIZE offsetof(lnet_ping_info_t, pi_ni[LNET_MAX_RTR_NIS])

Following the call chain from adding a new NI with DLC....

./lnet/utils/lnetconfig/liblnetconfig.c:
l_ioctl(LNET_DEV_ID, IOC_LIBCFS_ADD_NET, &data)

./lnet/lnet/module.c:
l_ioctl() -> lnet_dyn_configure() -> lnet_dyn_add_ni()

lnet/lnet/api-ni.c:
lnet_dyn_add_ni() -> lnet_ping_info_setup() -> lnet_ping_info_create()

This is where the pi_ni array mentioned above is used:

lnet_ping_info_create(int num_ni)
...
infosz = offsetof(lnet_ping_info_t, pi_ni[num_ni]);
…

This limit would appear designed to apply to routers, but what about an OSS with an arbitrary number (N) of NI's (e.g. o2ib0…o2ibN)?

Issue Links

• is duplicated by: LU-7562 Increase LNET_MAX_INTERFACES
• is related to: LU-6901 LNet: Support dynamic number of NIDs in ping response
• is related to: LU-9680 Improve the user land to kernel space interface for lustre