Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14288

Enhance nodemap ranges to work better with IPv6

Details

    • Improvement
    • Resolution: Fixed
    • Major
    • Lustre 2.17.0
    • Lustre 2.16.0
    • 9223372036854775807

    Description

      The nodemap functionality in lustre allows sets of hosts to be described using a range of network addresses.

      For network types where the address is a simple integer, this seems reasonable.

      For IPv4 where an address is usually represented as 4 small integers, this is manageable, but not really ideal.  The standard throughout IP is to use a netmask to identify a set of addresses, whether for routing, for access control, or any other purpose.

      For IPv6 it would be quite inconvenient to use ranges.  Addresses are rarely allocated sequentially, and specifying a netmask length is much easier.  Also, it is standard practice to store IPv6 address (and IPv4) in network-byte-order, so comparing two addresses to see if they are ordered (as needed for ranges) is clumsy to implement.

      Clearly we need to keep address-range support for existing address types, but I believe we should not support it for new IP address.  Further we should add netmask support for existing address types.

      Internally, this would mean keeping nodemap information in a range-tree for 4-bytes addresses (as we already do), and using a netmask based structure for longer addresses.

      From user-space, we would add a syntax (/nn suffix) to specify a netmask.  For 4-byte addresses this would be converted to a range.  For large address, this would be used as-is.

       

      Attachments

        Issue Links

          Activity

            [LU-14288] Enhance nodemap ranges to work better with IPv6
            pjones Peter Jones added a comment -

            Merged for 2.17

            pjones Peter Jones added a comment - Merged for 2.17

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56184/
            Subject: LU-14288 nodemap: Use nidmasks for IPv6 NIDs
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 8f6988be4416fe0256583e5d6ea63e92ec0811ca

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56184/ Subject: LU-14288 nodemap: Use nidmasks for IPv6 NIDs Project: fs/lustre-release Branch: master Current Patch Set: Commit: 8f6988be4416fe0256583e5d6ea63e92ec0811ca

            We have one patch in gerrit but I still need to create a patch to move from linked list to interval tree. The work for using interval tree with jobstats by Shuan will provide the needed bits to implement proper interval trees for nodemap

            simmonsja James A Simmons added a comment - We have one patch in gerrit but I still need to create a patch to move from linked list to interval tree. The work for using interval tree with jobstats by Shuan will provide the needed bits to implement proper interval trees for nodemap

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55922/
            Subject: LU-14288 lnet: Introduce nidmasks
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 4b12a9dcaf7ef08ec37c9209c31e44623f57548c

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55922/ Subject: LU-14288 lnet: Introduce nidmasks Project: fs/lustre-release Branch: master Current Patch Set: Commit: 4b12a9dcaf7ef08ec37c9209c31e44623f57548c

            "Chris Horn <chris.horn@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56184
            Subject: LU-14288 nodemap: Use nidmasks for IPv6 NIDs
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: a6e64cb3ee208ec12aad59c434ea2922d72f7232

            gerrit Gerrit Updater added a comment - "Chris Horn <chris.horn@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56184 Subject: LU-14288 nodemap: Use nidmasks for IPv6 NIDs Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: a6e64cb3ee208ec12aad59c434ea2922d72f7232

            "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56058
            Subject: LU-14288 nodemap: debug IPv6 failures
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 91b195645b2d233cb67b2b8fecd17783d2accdb7

            gerrit Gerrit Updater added a comment - "James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56058 Subject: LU-14288 nodemap: debug IPv6 failures Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 91b195645b2d233cb67b2b8fecd17783d2accdb7

            In our latest testing for sanity-sec we are seeing a kernel crash for IPv6 with nodemap. I can't reproduce it locally so its something WC testbed specific.

            simmonsja James A Simmons added a comment - In our latest testing for sanity-sec we are seeing a kernel crash for IPv6 with nodemap. I can't reproduce it locally so its something WC testbed specific.
            hornc Chris Horn added a comment -

            I've started plugging away at this. It is not a huge lift to modify the existing nid range code to support large NIDs. While it may not be practical to actually use this to specify lots of IPv6 NIDs, I think it is useful to have this support so that code can be simplified for cases where just a couple NIDs are specified (like various test cases). i.e. callers can pass any kind of nid to cfs_parse_nidlist and it "just works". I had originally had a few lines of code that would reject large nids that contained a range expression, but it seemed silly to prevent that when the rest of the code can handle them just fine.

            In any case, these patches are not done, but they are fairly well baked. I'd appreciate feedback as I continue to polish them.

            hornc Chris Horn added a comment - I've started plugging away at this. It is not a huge lift to modify the existing nid range code to support large NIDs. While it may not be practical to actually use this to specify lots of IPv6 NIDs, I think it is useful to have this support so that code can be simplified for cases where just a couple NIDs are specified (like various test cases). i.e. callers can pass any kind of nid to cfs_parse_nidlist and it "just works". I had originally had a few lines of code that would reject large nids that contained a range expression, but it seemed silly to prevent that when the rest of the code can handle them just fine. In any case, these patches are not done, but they are fairly well baked. I'd appreciate feedback as I continue to polish them.

            "Chris Horn <chris.horn@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55922
            Subject: LU-14288 lnet: Introduce nidmasks
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 3db9ca1439f5eb1a836efdcc7d3b8d830169f8e0

            gerrit Gerrit Updater added a comment - "Chris Horn <chris.horn@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55922 Subject: LU-14288 lnet: Introduce nidmasks Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 3db9ca1439f5eb1a836efdcc7d3b8d830169f8e0
            gerrit Gerrit Updater added a comment - - edited

            "Chris Horn <chris.horn@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55921
            Subject: LU-14288 libcfs: Support large NIDs in nidranges
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: e407cda19b98456b417bee3e347b16306eeab6ea

            gerrit Gerrit Updater added a comment - - edited "Chris Horn <chris.horn@hpe.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55921 Subject: LU-14288 libcfs: Support large NIDs in nidranges Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: e407cda19b98456b417bee3e347b16306eeab6ea

            People

              simmonsja James A Simmons
              neilb Neil Brown
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: