[LU-14288] Enhance nodemap ranges to work better with IPv6

• Type: Improvement
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.17.0
• Affects Version/s: Lustre 2.16.0
• Labels: IPv6
• Assignee: James A Simmons
• Reporter: Neil Brown

Description

The nodemap functionality in lustre allows sets of hosts to be described using a range of network addresses.

For network types where the address is a simple integer, this seems reasonable.

For IPv4 where an address is usually represented as 4 small integers, this is manageable, but not really ideal.  The standard throughout IP is to use a netmask to identify a set of addresses, whether for routing, for access control, or any other purpose.

For IPv6 it would be quite inconvenient to use ranges.  Addresses are rarely allocated sequentially, and specifying a netmask length is much easier.  Also, it is standard practice to store IPv6 address (and IPv4) in network-byte-order, so comparing two addresses to see if they are ordered (as needed for ranges) is clumsy to implement.

Clearly we need to keep address-range support for existing address types, but I believe we should not support it for new IP address.  Further we should add netmask support for existing address types.

Internally, this would mean keeping nodemap information in a range-tree for 4-bytes addresses (as we already do), and using a netmask based structure for longer addresses.

From user-space, we would add a syntax (/nn suffix) to specify a netmask.  For 4-byte addresses this would be converted to a range.  For large address, this would be used as-is.

 

Issue Links

• is related to: LU-18014 conf-sanity test_43a: @@@@@@ FAIL: check lustre-MDTall.mdt.nosquash_nids failed!
• is related to: LU-18167 sanity-sec test_16: nodemap_classify_nid()) ASSERTION( nodemap != ((void *)0) ) failed
• is related to: LU-18417 Finish IPv6 support
• is related to: LU-10391 LNET: Support IPv6
• is related to: LU-17217 Allow server to control/deny client connections
• is related to: LU-13307 add LCFG_NODEMAP_ADD_RANGEv6 records for IPv6

Comments

[James A Simmons]

We have one patch in gerrit but I still need to create a patch to move from linked list to interval tree. The work for using interval tree with jobstats by Shuan will provide the needed bits to implement proper interval trees for nodemap

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55922/
Subject: LU-14288 lnet: Introduce nidmasks
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4b12a9dcaf7ef08ec37c9209c31e44623f57548c

[Gerrit Updater]

"Chris Horn <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56184
Subject: LU-14288 nodemap: Use nidmasks for IPv6 NIDs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a6e64cb3ee208ec12aad59c434ea2922d72f7232

[Gerrit Updater]

"James Simmons <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56058
Subject: LU-14288 nodemap: debug IPv6 failures
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 91b195645b2d233cb67b2b8fecd17783d2accdb7

[James A Simmons]

In our latest testing for sanity-sec we are seeing a kernel crash for IPv6 with nodemap. I can't reproduce it locally so its something WC testbed specific.

[Chris Horn]

I've started plugging away at this. It is not a huge lift to modify the existing nid range code to support large NIDs. While it may not be practical to actually use this to specify lots of IPv6 NIDs, I think it is useful to have this support so that code can be simplified for cases where just a couple NIDs are specified (like various test cases). i.e. callers can pass any kind of nid to cfs_parse_nidlist and it "just works". I had originally had a few lines of code that would reject large nids that contained a range expression, but it seemed silly to prevent that when the rest of the code can handle them just fine.

In any case, these patches are not done, but they are fairly well baked. I'd appreciate feedback as I continue to polish them.

[Gerrit Updater]

"Chris Horn <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55922
Subject: LU-14288 lnet: Introduce nidmasks
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 3db9ca1439f5eb1a836efdcc7d3b8d830169f8e0

[Gerrit Updater]

"Chris Horn <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55921
Subject: LU-14288 libcfs: Support large NIDs in nidranges
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e407cda19b98456b417bee3e347b16306eeab6ea

[James A Simmons]

Andreas that is an interesting idea. Neil looking at this I was wondering to avoid reinventing the wheel does the kernel have any IPv6 / netmask -> collection of IPv6 address functions. 

[Andreas Dilger]

simmonsja, I was just wondering in the context of LU-17217 if it would make sense to extend this functionality to add a nodemap record that matches all IPv6 clients (i.e. LNET_NID_ANY), in addition to the current "match one IPv6 client"? That might be useful to block IPv6 clients from connecting, if we find out that they are causing some issue (e.g. crashing the server, or something).

That should be relatively easily implemented beside the current "match one" functionality without having to deal with how do we match ranges of IPv6 addresses.

[James A Simmons]

Now to phase II.