[LU-2563] Failure on test suite lustre-initialization-1: Invalid filesystem name

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.4.0
• Fix Version/s: Lustre 2.4.0
• Type: Bug
• Priority: Blocker
• Reporter: Maloo
• Assignee: Chris Gearing (Inactive)
• Resolution: Fixed
• Labels: HB, zfs
• Severity: 3

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/2b7215e4-551b-11e2-8753-52540035b04c.

The sub-test lustre-initialization_1 failed with the following error:

Test system failed to start single suite, so abandoning all hope and giving up

This error blocks ZFS testing

11:26:52:CMD: client-32vm3 mkfs.lustre --mgs --param=sys.timeout=20 --backfstype=zfs --device-size=2097152 --reformat /dev/lvm-MDS/P1 /dev/lvm-MDS/P1
11:26:52:client-32vm3: 
11:26:52:client-32vm3: mkfs.lustre FATAL: Invalid filesystem name /dev/lvm-MDS/P1
11:26:52:client-32vm3: 
11:26:52:client-32vm3: mkfs.lustre FATAL: unable to prepare backend (22)
11:26:52:client-32vm3: mkfs.lustre: exiting with 22 (Invalid argument)

Comments

[Di Wang]

Ah, t-f.sh has been changed for this part.

11:26:34:export MGSDEV=/dev/lvm-MDS/P1
11:26:34:export MDSDEV=/dev/lvm-MDS/P1

MGSDEV definition should be removed for combined mds/mgs test in the config of maloon test.

[Li Wei (Inactive)]

The TF change looks suspicious to me... The MGSDEV assignment, although unnecessary, should work.

[Li Wei (Inactive)]

Autotest was actually using a combined-MDT-MGT configuration. I think the combined_mds_mgs() call in formatall() returned false because mgsdevname() printed "/dev/lvm-MDS/P1" while mdsdevname() printed "lustre-mdt1/mdt1". mgsdevname() should print dataset names for ZFS-based MGTs.

[Li Wei (Inactive)]

http://review.whamcloud.com/4950

[Andreas Dilger]

Patch landed to master, closing bug.

Li Wei, in the future please submit patches that affect ZFS with:

Test-Parameters: mdtfilesystemtype=zfs ostfilesystemtype=zfs

so that we are able to verify that this is working correctly with ZFS.

[Li Wei (Inactive)]

Sure. I thought Autotest would start a ZFS job automatically. On the other hand, ZFS testing was already broken. Landing this fix won't make things any worse.

[Andreas Dilger]

Looks like the patch that was landed is still broken, per messages in the autotest logs:

https://maloo.whamcloud.com/test_sets/04d030c6-6734-11e2-95b6-52540035b04c

12:09:44:/usr/lib64/lustre/tests/test-framework.sh: line 2690: [: /dev/lvm-MDS/P1: unary operator expected

which corresponds to the modified code:

99baa1523 2688)   zfs )
99baa1523 2689)        if [ $(facet_host mgs) = $(facet_host mds1) ] &&
99baa1523 2690)              ( [ -z "$MGSDEV" ] || [ $MGSDEV = $(mdsvdevname 1) ] ); then
99baa1523 2691)                   DEVPTR=$(mdsdevname 1)

This is causing all ZFS testing to fail.

Minor style note - there is no reason to invoke multiple separate "test" ("[") conditions here:

        zfs)
                if [[ $(facet_host mgs) = $(facet_host mds1) &&
                     ( -z "$MGSDEV" -p "$MGSDEV" = "$(mdsdevname1)" ) ]]; then

[Li Wei (Inactive)]

I don't think it is because of the patch above. (I had tested it locally with both combined and separate MGT/MDT as well as ldiskfs and ZFS.) Judging from the error message, mdsvdevname() printed an empty string.

[Li Wei (Inactive)]

Chris, it looks the inconsistency in Autotest configuration is back:

12:09:14:export mgs_HOST=client-30vm7
12:09:14:export mds_HOST=client-30vm7
12:09:14:export MGSDEV=/dev/lvm-MDS/P1
12:09:14:export MDSDEV=/dev/lvm-MDS/P1
[...]
12:09:15:export MDSFSTYPE=ldiskfs
12:09:15:export MGSFSTYPE=zfs

The devices and hosts definitions suggest a combined MGT/MDT, but the fstypes contradict each other. Could you please fix that?

[Li Wei (Inactive)]

P.S., unless we intentionally want mixed-fstype configurations, Autotest could only set "FSTYPE=zfs", leaving XXXFSTYPEs undefined.

[Chris Gearing (Inactive)]

I don't intend to move to FSTYPE, the definitions of the three separately should work.

In the first example mds,met,oat filesystemtype are all zfs.

In the second example that Li Wei refers to they are incorrect because they are incorrectly specified in the test parameters in the review message.

mdtfilesystemtype=zfs ostfilesystemtype=zfs

are specified but mdsfilesystemtype=zfs is not.

I've added a comment to the review, and resubmitted it with the additional definition.

[Li Wei (Inactive)]

I didn't notice the "Test-Parameters" line in the session reported by Andreas and incorrectly assumed Autotest had all controls on the generated configuration. Sorry about that.

As to "mdsfilesystemtype", I'm not sure how it differs from "mdtfilesystemtype". Actually, I asked this on the wiki quite a while ago. It looks like a typo of "mgsfilesystemtype", but your comment makes me think it is not. Could you clarify, please?

[Andreas Dilger]

Li Wei, I agree - the Test-Parameter should be named "mgsfilesystemtype", NOT "mdsfilesystemtype".

I would also be happy if there was a shorthand, like "filesystemtype=zfs" so that the verbose (and potentially failure-inducing) separate parameters can be avoided for the time being.

That there is unquestionably still a bug somewhere, unrelated to the Test-parameters, because in the following test the FSTYPE is specified correctly, but the MGS still fails during mkfs.lustre:

https://maloo.whamcloud.com/test_logs/7762f79c-6bf6-11e2-b2a5-52540035b04c

13:30:38:export MGSDEV=/dev/lvm-MDS/P1
13:30:38:export MDSDEV=/dev/lvm-MDS/P1
13:30:38:export MDSFSTYPE=zfs
13:30:38:export MGSFSTYPE=zfs
13:30:52:CMD: wtm-26vm7 mkfs.lustre --mgs --param=sys.timeout=20 --backfstype=zfs --device-size=2097152 --reformat /dev/lvm-MDS/P1 /dev/lvm-MDS/P1
13:30:52:wtm-26vm7: 
13:30:52:wtm-26vm7: mkfs.lustre FATAL: Invalid filesystem name /dev/lvm-MDS/P1

Nothing is on the MDS or client console or debug logs.

[Li Wei (Inactive)]

Emoly's patch is based on a Jan 17 master, which does not include the fix above. It looks like a result of the original issue. I'll ask her to rebase if necessary.

[Andreas Dilger]

Test failure was run without the patch that should fix this. Closing bug again.