[LU-4334] With ZFS can only declare a single mgsnode for MDT or OST

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.7.0, Lustre 2.5.4
• Affects Version/s: Lustre 2.4.1, Lustre 2.6.0, Lustre 2.5.2
• Labels: llnl prz zfs
• Environment: SL6.4

Epic/Theme

zfs

• Severity: 1
• Assignee: Nathaniel Clark
• Reporter: Eric Kolb

Description

When trying to declare two mgsnodes for HA only one entry is ever accepted when using backfstype=zfs. All below examples fail, if the --mgsnode=nid:nid syntax worked then all might be okay.

mkfs.lustre --reformat --fsname=RSF1 --ost --index=2 --mgsnode=10.82.0.9@tcp1 --mgsnode=10.82.0.9@tcp1:10.82.0.10@tcp1 --servicenode=10.82.0.11@tcp1:10.82.0.12@tcp1 --backfstype=zfs OST2/ost

mkfs.lustre --reformat --fsname=RSF1 --ost --index=2 --mgsnode=10.82.0.9@tcp1:10.82.0.10@tcp1 --servicenode=10.82.0.11@tcp1:10.82.0.12@tcp1 --backfstype=zfs OST2/ost

Issue Links

• is related to: LU-4749 ZFS-backed OST mkfs.lustre --servicenode does not correctly add failover_nids
• is related to: LU-8311 Target does not mount with the new mgsnode parameter format in case of multirail configuration

Comments

[Chris Gearing (Inactive)]

Does

zfs set lustre:mgsnode=192.168.139.10@tcp:192.168.139.70@tcp mdt/mdt1 

imply that ':' is the standard separator for zfs properties, for example in the LU-4749 does failnode be split the same way.

[Peter Jones]

Landed for 2.7

[Nathaniel Clark]

This will fix setting multiple mgsnode properties on ZFS.
http://review.whamcloud.com/11161

Workaround for older systems:

Instead of

tunefs.lustre --mgsnode=192.168.139.10@tcp --mgsnode=192.168.139.70@tcp mdt/mdt1

Use the following:

zfs set lustre:mgsnode=192.168.139.10@tcp:192.168.139.70@tcp mdt/mdt1 

[Nathaniel Clark]

It seems like the right idea would be to store NID information in a single ZFS property <server1ip1>@tcp,<server1ip2>@tcp:<server2ip1>@tcp,<server2ip2>@tcp similar to how it can be input on the command line.

This will apply to mgsnode, failnode, and servicenode.

[Christopher Morrone (Inactive)]

As part of fixing this issue, we need to make certain that the relevant OSD documentation is updated to clearly define the APIs and expecations for the OSD developer.

[Nathaniel Clark]

It looks like there are two separate issues here:

1) listing failover or mgsnode in the form --mgsnode=NID1 --mgsnode=NID2 or --mgsnode=NID1,NID2 will result in only NID2 being recorded.

This seems to be due to how metadata is stored on zfs, that property names are unique, thus setting it twice will just overwrite the first with the second.

2) listing nids in the form --mgsnode=NID1:NID2 will result in only NID1 being used