[LU-10930] lnet module parameter "forwarding" does not enable routing

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.11.0
• Type: Bug
• Priority: Minor
• Reporter: Olaf Faaland
• Assignee: Sonia Sharma (Inactive)
• Resolution: Unresolved
• Labels: llnl

Environment

Red Hat Enterprise Linux Server release 7.5 Beta (Maipo)
Linux 4.14.0-43.el7a.aarch64

• Severity: 3

Description

options lnet forwarding="enabled"

Setting forwarding="enabled" in lnet modprobe conf file does not enable routing.  That is, after starting lnet with the configuration below, "lnetctl routing show" returns nothing.  Also, a compute node (also 2.11) with a route whose gateway is configured as below, shows the route is down.

The specific configuration tested was:

[root@ulna4:~]# cat /etc/modprobe.d/lnet.conf
options lnet forwarding="enabled"

and

[root@ulna4:~]# cat /etc/lnet.conf
ip2nets:

• net-spec: o2ib100
  interfaces:
  0: san0
  ip-range:
  0: 172.19.1.*
• net-spec: o2ib33
  interfaces:
  0: hsi0
  ip-range:
  0: 192.168.128.*
  route:
  <routes redacted>

 

Comments

[Olaf Faaland]

I haven't tested a combination of lnetctl import/YAML configuration and lnet module options configuration like this, in 2.10 yet.

[Peter Jones]

Sonia

Can you please advise?

Thanks

Peter

[Amir Shehata (Inactive)]

There is a couple of ways you can start up LNet. The legacy way is lctl net up. And the new way is lnetctl lnet configure.

When you run lnetctl lnet configure. It doesn't configure the system using any of the module parameters which lnetctl handles. This was by design to prevent conflict between module parameters and lnet.conf YAML file.

If you would like to load the module parameters you can run: lnetctl lnet configure --all.

I see though that you are using lnet.conf to startup the system, so I wouldn't recommend loading the module parameters which are also configured from lnetctl.

The idea is to rely on only one source of configuration.

[Olaf Faaland]

> The idea is to rely on only one source of configuration.

That makes sense, but then I think at least two changes are necessary:

1. Lustre should report it when module options are provided but are ignored.
2. The policy you describe should be  loud and proud in the man pages.

Also, lnetctl routing show and lnetctl export need to report that routing is disabled.  As you see below, neither do, although they report when it is enabled.

[faaland1@ulna66 ~] $sudo lnetctl routing show
[faaland1@ulna66 ~] $sudo lnetctl export | grep -C3 routing
[faaland1@ulna66 ~] $

 

 

[Olaf Faaland]

I verified that with 2.11, if "forwarding" and "networks" are set as module parameters, that lnetctl configure --all does enable routing as well as creating networks and associated NIDs.

Also, thanks for the clarification.