[LU-15091] Trying to start OBD ls3-MDT0000_UUID using the wrong disk ls30000_UUID. Were the /dev/ assignments rearranged

• Type: Bug
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.14.0
• Labels: llnl
• Environment: zfs-2.1.0_1llnl ; lustre-2.14.0_5.llnl ; 4.18.0-305.7.1.1toss.t4.x86_64 ; rhel 8.4
• Severity: 3
• Assignee: Yang Sheng
• Reporter: Olaf Faaland

Description

After renaming a file system and updating NIDs on the targets, MDT0000 fails to mount with the following error:

LustreError: 157-3: Trying to start OBD ls3-MDT0000_UUID using the wrong disk ls30000_UUID. Were the /dev/ assignments rearranged?

Note that lsd->lsd_uuid is missing "-MDT" between the fs name ("ls3") and the MDT index ("0000").

The rename was probably accomplished with:

tunefs.lustre --writeconf --fsname=ls3 --rename=lustre3 -v asp1/mdt1

And the NID update was probably accomplished with:

tunefs.lustre --param=mgsnode=172.19.1.141@o2ib100:172.19.1.142@o2ib100 --param=failover.node=172.19.1.141@o2ib100:172.19.1.142@o2ib100 asp1/mdt1

Unfortunately I no longer have the output from those commands, and I'm not certain exactly when this occurred.

This only occurred on one MDT out of 12 targets (4 MDT 8 OST). I don't know why this one was different.

I don't think this is enough information to find the root cause and fix it, but am creating the issue in hopes it prompts anyone else who sees this issue to document what led up to it.