Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-16959

LNet: auto-tune ARP-related sysctl setting

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      Default linux settings for net.ipv4.neigh.default.gc_thresh* may be too low:

      net.ipv4.neigh.default.gc_thresh1 = 128
      net.ipv4.neigh.default.gc_thresh2 = 512
      net.ipv4.neigh.default.gc_thresh3 = 1024

      For example, this can cause the garbage collector in the kernel to start deleting entries in the table once the table size exceeds gc_thresh2, leading to unnecessary ARP traffic.

      If IP to MAC mappings in the system are permanent, setting a higher gc_thresh1 may also help avoid unnecessary ARP traffic as this is the threshold for when entries start timing out. 

      In large systems this may cause troubles with evictions, etc.

      The memory cost of having larger tables is low, so It is proposed to use the following settings:

      net.ipv4.neigh.default.gc_thresh1 = 8192
      net.ipv4.neigh.default.gc_thresh2 = 16384
      net.ipv4.neigh.default.gc_thresh3 = 32768

      The user should be able to make the choice to manage the settings on their own, probably via a module parameter.

      Note that there's the similar set of IPv6 parameters to consider.

      Attachments

        Activity

          People

            fsehr Frank Sehr
            ssmirnov Serguei Smirnov
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: