Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-6130

Number of LNET NI's limited

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • Lustre 2.7.0
    • None
    • 3
    • 17076

    Description

      While investigating the number of maximum possible NI's that can be added with DLC, I found that the structure lnet_ping_info_t is only expected to have a maximum of 16 entries in pi_ni.

      ./lnet/include/lnet/lib-types.h:460

      typedef struct {
      __u32 pi_magic;
      __u32 pi_features;
      lnet_pid_t pi_pid;
      __u32 pi_nnis;
      lnet_ni_status_t pi_ni[0];
      } WIRE_ATTR lnet_ping_info_t;

      /* router checker data, per router */
      #define LNET_MAX_RTR_NIS 16
      #define LNET_PINGINFO_SIZE offsetof(lnet_ping_info_t, pi_ni[LNET_MAX_RTR_NIS])

      Following the call chain from adding a new NI with DLC....

      ./lnet/utils/lnetconfig/liblnetconfig.c:
      l_ioctl(LNET_DEV_ID, IOC_LIBCFS_ADD_NET, &data)

      ./lnet/lnet/module.c:
      l_ioctl() -> lnet_dyn_configure() -> lnet_dyn_add_ni()

      lnet/lnet/api-ni.c:
      lnet_dyn_add_ni() -> lnet_ping_info_setup() -> lnet_ping_info_create()

      This is where the pi_ni array mentioned above is used:

      lnet_ping_info_create(int num_ni)
      ...
      infosz = offsetof(lnet_ping_info_t, pi_ni[num_ni]);

      This limit would appear designed to apply to routers, but what about an OSS with an arbitrary number (N) of NI's (e.g. o2ib0…o2ibN)?

      Attachments

        Issue Links

          Activity

            People

              core-lustre-triage Core Lustre Triage
              blakecaldwell Blake Caldwell
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: