[LU-10991] unable to import pools : Upgrading 2.10.3 to 2.11.51

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.12.0
• Type: Bug
• Priority: Critical
• Reporter: Saurabh Tandan (Inactive)
• Assignee: Nathaniel Clark
• Resolution: Cannot Reproduce

Environment

Older version: 2.10.3
New version: 2.11.51
ZFS
2 clients, 1 OSS with 2 OSTs, 1 MDS with 1 MDT

• Severity: 3

Description

Unable to import pools when upgrading OSS from 2.10.3 to 2.11.51.

Steps:

All clients and servers set up with 2.10.3 lustre version.

2.10.3 ZFS version: zfs-0.7.5-1.el7

2.11.51 ZFS version: zfs-0.7.8-1.el7

Upgraded OSS to 2.11.51 and tried to import pools but getting the following:

[root@onyx-70 ~]# modprobe zfs

zpool import lustre-ost2[ 5805.811988] spl: loading out-of-tree module taints kernel.

[ 5805.818394] spl: module verification failed: signature and/or required key missing - tainting kernel

[ 5805.834983] SPL: Loaded module v0.7.8-1

[ 5805.839599] znvpair: module license 'CDDL' taints kernel.

[ 5805.845676] Disabling lock debugging due to kernel taint

[ 5807.838597] ZFS: Loaded module v0.7.8-1, ZFS pool version 5000, ZFS filesystem version 5

[root@onyx-70 ~]# 

[root@onyx-70 ~]# zpool import lustre-ost1

cannot import 'lustre-ost1': no such pool available

[root@onyx-70 ~]# zpool import lustre-ost2

cannot import 'lustre-ost2': no such pool available

[root@onyx-70 ~]# zpool list

no pools available

Issue Links

• is related to: LU-10998 Add 2.10.0 zfs and ldiskfs images for...

Comments

[Andreas Dilger]

Please also create new test images for conf-sanity test_32. 

[James Nunez (Inactive)]

LU-10998 is tracking creating new images for conf-sanity test 32.

[Andreas Dilger]

Was this resolved with "zfs import -a"?

[Saurabh Tandan (Inactive)]

Yes it was while doing rolling upgrade/downgrade. But while doing clean upgrade/downgrade I tried to import it using both pool name and "-a" it didn't worked either way.

[Sarah Liu]

The cannot import pool failure still exists when clean upgrade from 2.10.5 to 2.11.56. Both builds use zfs-0.7.9-1

[Sarah Liu]

Just tried upgrade from 2.11.0 to master, not seeing this problem, pool can be imported after upgrade.

[Andreas Dilger]

The cannot import pool failure still exists when clean upgrade from 2.10.5 to 2.11.56. Both builds use zfs-0.7.9-1

Sarah,
are there any error messages on the console when you try to import the pool? This would be a fairly serious error if we can't upgrade from 2.10.x (which is the version almost every ZFS user is using) to 2.12 

[Andreas Dilger]

Nathaniel, can you work with Sarah to understand this issue better? It might be related to some new ZFS feature we are trying to use? In any case, we need to understand it before 2.12 goes out the door or we could have a big compatibility problem on our hands. Hopefully the issue is not already present in the 2.11 release, but at least relatively few people are using that.

[Nathaniel Clark]

The zpool import should be independent of the Lustre version. Do you have a log (messages and journal)?
If you still have a system, can you also run:

zpool events -v

[Sarah Liu]

There isn't error mesg, just a message says "no pool available"

I will run the command Nathaniel mentioned to get more info.

[Sarah Liu]

I cannot reproduce the problem with 2 trials (1 with fresh format zfs; 2 with ldiskfs first, then reformat as zfs); I think there must be some misoperation in my previous test.

[Peter Jones]

ok - thanks Sarah!