[LU-8912] nodemap: error: nodemap_add_range: nodemap ranges must be contiguous

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.9.0
• Fix Version/s: Lustre 2.11.0, Lustre 2.10.4
• Type: Bug
• Priority: Minor
• Reporter: Stephane Thiell
• Assignee: Kit Westneat
• Resolution: Fixed

Environment

CentOS 7.2, Lustre-2.8.60

• Severity: 3

Description

Hi, we noticed the following minor issue when trying to add our IB network 10.210.32.0/20 to nodemap:

$ lctl nodemap_add_range --name sherlock --range 10.210.[32-47].[0-255]@o2ib3
error: nodemap_add_range: nodemap ranges must be contiguous
$ cat /proc/fs/lustre/nodemap/sherlock/ranges
[

]

The workaround is either to add 16 separate entries (10.210.32.[0-255], 10.210.33.[0-255] etc.), or a single larger range (whenever possible).

Thanks,
Stephan

Comments

[Peter Jones]

Kit

Could you please advise on this issue?

Thanks

Peter

[Kit Westneat]

I'm not sure how the contiguous ranges is figured out, I'll try to see how hard it would be to get this to work. 

[Kit Westneat]

It looks like the LNET range parsing is currently not complete, and broken in parts. It would be much easier to use a subset of the range. The full LNET range syntax supports multiple NID expressions and multiple range expressions within the NID expressions. For example, these are both valid:
lctl nodemap_add_range --name c0 --range "10.210.1.[0-255]@tcp 10.210.2.0@tcp"
lctl nodemap_add_range --name c0 --range "10.210.1.[33,31,32]@tcp"

It would be significantly easier to parse if we only allow a single NID expression per range, and a single range expression per IP expression list.

Alternatively, lctl actually just converts the range syntax into a simpler one read by the MGS, which is just <first nid>:<last nid>. Would it be too confusing to have sysadmins use that directly?

[Stephane Thiell]

Hi Kit,

Thanks for looking at this! It would be better if the sysadmin can just add <first nid>:<last nid>, as it is already the format used internally by the MGS and also used when accessing the nodemap range config (using /proc/fs/lustre/nodemap/sherlock/ranges for example). I am sure it would be less confusing after all, along with the restrictions of range syntax that you propose to avoid any mistakes.

[Kit Westneat]

I tend to agree with you, but on our nodemap call, Intel voiced a strong preference for supporting the full LNET range expression support. I've prepared a bug fix patch to address your immediate issue.

The idea behind the full LNET range support is that instead of requiring non-contiguous ranges, if a non-contiguous range is given to lctl, multiple contiguous ranges will be created in the backend. I'll create an improvement ticket to track the work on that.

[Gerrit Updater]

Kit Westneat ([e-mail]) uploaded a new patch: https://review.whamcloud.com/24397
Subject: LU-8912 nodemap: fix contiguous range support
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f9bf47f8590cdd77a7b896eea7e83e91782f04bd

[Minh Diep]

kit.westneat, please rebase the patch manually. thanks

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/24397/
Subject: LU-8912 nodemap: fix contiguous range support
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: eac95a6587af4061b36b35051e191ca27bc6b1a3

[Peter Jones]

Landed for 2.11

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31521
Subject: LU-8912 nodemap: fix contiguous range support
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: 1f9c0431a29e2f0a8cac90eef9ae0dfa18b02cae

[Gerrit Updater]

John L. Hammond ([e-mail]) merged in patch https://review.whamcloud.com/31521/
Subject: LU-8912 nodemap: fix contiguous range support
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: 92d562457b947f02891e7c76b265bde4f9ed23c2