Details
-
Improvement
-
Resolution: Unresolved
-
Minor
-
None
-
Lustre 2.16.0, Lustre 2.15.2
-
None
-
9223372036854775807
Description
When the MGS has multipe failover hosts, and each host has multiple NIDs, the mount command line can become excessively long:
# mount -t lustre 172.16.0.26@o2ib,172.16.0.27@o2ib:172.16.0.30@o2ib,172.16.0.31@o2ib:172.16.0.28@o2ib,172.16.0.29@o2ib:172.16.0.24@o2ib,172.16.0.25@o2ib:/testfs /lustre/testfs/client
Not only is this very inconvenient to type and prone to error, but it is also unsightly in "df" and "mount" output because of the very long "device" name. This will become significantly worse once IPv6 NIDs are in use (each addresse will be about twice as long and hard to visually parse), and may even limit the number of NIDs that can be specified on the command-line due to the length.
It is already possible (though uncommonly used, for some reason) to use DNS hostname lookup for the MGS NID in the case of TCP hostnames. The mount.lustre command will do DNS hostname resolution and pass the numeric NIDs to the kernel:
# mount -t lustre mgs-fs1-1a@o2ib,mgs-fs1-1b@oi2b:mgs-fs1-2a@o2ib,mgs-fs1-2b@o2ib:mgs-fs1-3a@o2ib,mgs-fs1-3b@oi2b:mgs-fs1-4a@oi2b,mgs-fs1-4b@o2ib:/testfs /lustre/testfs/client
This is an improvement, but still requires that all of the MGS hostnames are supplied at mount, but at least avoids the need for manually specifying all of the IP addresses. If the network type is not specified, then it defaults to @tcp0, but it is possible to use @o2ib since this will also resolve to an IP address.
A few improvements could be made to shorten this command-line:
- allow host or NID ranges to be specified:
# mount -t lustre 172.16.0.[24-30/2]@o2ib,172.16.0.[25-31/2]@o2ib:/testfs /lustre/testfs/client # mount -t lustre mgs-fs1-[1-4]a@o2ib,mgs-fs1-[1-4]b@o2ib:/testfs /lustre/testfs/client
This will be significantly less to type than the full NID list, but is somewhat opaque and would still need to be expanded internally by mount.lustre to the full NID list for passing to the kernel. The full list would still be visible in the "mount" and "df" output.
- allow round-robin DNS for the MGS interfaces from a single hostname would avoid the need to specify multiple MGS hostnames on the command-line:
# host mgs-fs1 mgs-fs1 has address 141.193.213.10 mgs-fs1 has address 141.193.213.11 # mount -t mgs-fs1@o2ib:/testfs /lustre/testfs/client
This should work for both IPv4 and IPv6 hostname resolution, but requires that mount.lustre and/or libcfs_str2nid() are able to resolve a single name to multiple NIDs.
All of the "mount.lustre" improvements only shorten the input name given on the command line. However, /proc/mounts would still show the full list of MGS NIDs for df and mount output.
One (possibly controversial) option would be to shorten the command-line displayed in /proc/mounts to only list the NID(s) of the currently-active MGS. That would keep the command-line down to ~2 NIDs, but would mean that copying the shortened displayed command-line to other nodes would potentially cause them problems if the MGS failed over, because they would only know one set of NIDs.
Another option would be (in the case of hostnames or round-robin DNS) would be to internally pass the original MGS name as an "MGS display name" as an option the kernel mount(2) command (e.g. "--mgs-display=mgs-cli1@o2ib"). This would be shown by /proc/mounts and should work equally well for other clients, since they should also be able to resolve the hostname similarly.
