[LU-2456] Dynamic LNet Config Main Development Work

• Type: New Feature
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.7.0
• Affects Version/s: Lustre 2.6.0
• Assignee: Amir Shehata (Inactive)
• Reporter: Doug Oucharek (Inactive)

Description

This ticket has been created to track the main development work for the Dynamic LNet Config project.

Issue Links

• is duplicated by: LU-1071 able to handle many lnet routes
• is related to: LU-5734 LNet dynamic control: lnet_dyn_add_ni() can't clean up failed NI in some cases
• is related to: LU-5568 kernel crash when when network initialization failed
• is related to: LU-5838 DLC: inconsistent nameing of seqno vs seq_no
• is related to: LU-5849 DLC: parse networks function needs to return the number of parsed networks
• is related to: LU-5854 DLC: lnetctl import failed as Failed to handle token:2 [state=0, rc=-1]
• is related to: LU-5861 DLC: lnet startup script doesn't invoke lnetctl properly
• is related to: LU-5874 DLC: the ongoing traffic was interrupted after adding a new network interface
• is related to: LU-5875 DLC: failed adding an existing network interface when there is traffic ongoing
• is related to: LU-5884 bad lnet conf causes LBUG
• is related to: LUDOC-204 Update Lustre Manual for Dynamic LNET Config
• is related to: LU-5098 /proc/sys/lnet/routes is stupid
• is related to: LU-5850 DLC: lnet_startup_lndnis should clean up after itself
• is related to: LU-1279 failure trying to mount two targets at the same time after boot

Comments

[Amir Shehata (Inactive)]

Landed in 2.7

[Sarah Liu]

Test plan attached is dated, please update the document.

[Doug Oucharek (Inactive)]

Ticket LU-5712 was created to look at creating a new LGPL library from the parts of libcfs (GPL) used by this feature.

[James A Simmons]

Ticket LU-5705 was created to discuss lctl versus lnetctl

[Andreas Dilger]

One question that came up at LAD last week was why the lnetctl functionality was not (also) available via lctl? Like the other LNet-specific commands that are already available via lctl, these commands could also be added just by adding them to the lctl parser.

It is still desirable to have a standalone lnetctl command for the systems that are using only LNet (there are a few).

The stated benefit of having these new commands in lctl is that this reduces the number of binaries that need to be installed on the compute nodes, which try to have a minimal memory/space footprint.

That isn't a requirement for landing, but rather an enhancement request from the end users.

[Amir Shehata (Inactive)]

lnetctl utility has the same work flow as the ip utility. The only difference is that the infrastructure I was asked to use for the utility allows interactive mode, and the patter "help <cmd>" in interactive mode. However, that infrastructure doesn't support the pattern "help <cmd> <subcmd>" readily.

The way it works:

lnetctl > help set
set: set {tiny_buffers | small_buffers | large_buffers | routing}

lnetctl > set tiny_buffers help
set tiny_buffers: set tiny routing buffers
        VALUE must be greater than 0

so help <cmd> gives the first level of help. To get help on subsequent commands, you put the "help" keyword after the subcommand.

If you run directly from the command line, the output would look like this:

[root@rt1 lutfTMP]# lnetctl help
Available commands are:
        lnet
        route
        net
        routing
        set
        import
        export
        stats
        peer_credits
        help
        exit
        quit
For more help type: help command-name
[root@rt1 lutfTMP]# lnetctl route help
route add: add a route
        --net: net name (ex tcp0)
        --gateway: gateway nid (ex 10.1.1.2@tcp)
        --hop: number to final destination (1 < hops < 255)
        --priority: priority of route (0 - highest prio

route del: delete a route
        --net: net name (ex tcp0)
        --gateway: gateway nid (ex 10.1.1.2@tcp)

route show: show routes
        --net: net name (ex tcp0) to filter on
        --gateway: gateway nid (ex 10.1.1.2@tcp) to filter on
        --hop: number to final destination (1 < hops < 255) to filter on
        --priority: priority of route (0 - highest prio to filter on
        --detail: display detailed output per route

route help: display this help

[root@rt1 lutfTMP]# lnetctl route show help
route show: show routes
        --net: net name (ex tcp0) to filter on
        --gateway: gateway nid (ex 10.1.1.2@tcp) to filter on
        --hop: number to final destination (1 < hops < 255) to filter on
        --priority: priority of route (0 - highest prio to filter on
        --detail: display detailed output per route

[James A Simmons]

I have been testing this the utilities and I saw this:

lnetctl > help set
set: set

{tiny_buffers | small_buffers | large_buffers | routing}

lnetctl > help set large_buffers
setlarge_buffers: Unknown command.

Is this correct behavior?

[James A Simmons]

http://review.whamcloud.com/#/c/9830
http://review.whamcloud.com/#/c/9831
http://review.whamcloud.com/#/c/9832

http://review.whamcloud.com/#/c/8021
http://review.whamcloud.com/#/c/8022
http://review.whamcloud.com/#/c/8023
http://review.whamcloud.com/#/c/8025
http://review.whamcloud.com/#/c/8026
http://review.whamcloud.com/#/c/8027
http://review.whamcloud.com/#/c/8028

[Liang Zhen (Inactive)]

suggestions about dlc APIs based on discussion in Miami

[Liang Zhen (Inactive)]

comments for DLC patch http://review.whamcloud.com/#/c/9785/