Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-6130

Number of LNET NI's limited

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • Lustre 2.7.0
    • None
    • 3
    • 17076

    Description

      While investigating the number of maximum possible NI's that can be added with DLC, I found that the structure lnet_ping_info_t is only expected to have a maximum of 16 entries in pi_ni.

      ./lnet/include/lnet/lib-types.h:460

      typedef struct {
      __u32 pi_magic;
      __u32 pi_features;
      lnet_pid_t pi_pid;
      __u32 pi_nnis;
      lnet_ni_status_t pi_ni[0];
      } WIRE_ATTR lnet_ping_info_t;

      /* router checker data, per router */
      #define LNET_MAX_RTR_NIS 16
      #define LNET_PINGINFO_SIZE offsetof(lnet_ping_info_t, pi_ni[LNET_MAX_RTR_NIS])

      Following the call chain from adding a new NI with DLC....

      ./lnet/utils/lnetconfig/liblnetconfig.c:
      l_ioctl(LNET_DEV_ID, IOC_LIBCFS_ADD_NET, &data)

      ./lnet/lnet/module.c:
      l_ioctl() -> lnet_dyn_configure() -> lnet_dyn_add_ni()

      lnet/lnet/api-ni.c:
      lnet_dyn_add_ni() -> lnet_ping_info_setup() -> lnet_ping_info_create()

      This is where the pi_ni array mentioned above is used:

      lnet_ping_info_create(int num_ni)
      ...
      infosz = offsetof(lnet_ping_info_t, pi_ni[num_ni]);

      This limit would appear designed to apply to routers, but what about an OSS with an arbitrary number (N) of NI's (e.g. o2ib0…o2ibN)?

      Attachments

        Issue Links

          Activity

            People

              ashehata Amir Shehata (Inactive)
              blakecaldwell Blake Caldwell
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: