[LU-7695] zpools not available after first boot

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Labels: soak
• Environment: lola; build: https://build.hpdd.intel.com/job/lustre-reviews/36922/
• Severity: 3
• Assignee: Nathaniel Clark
• Reporter: Frank Heckes (Inactive)

Description

Error happened during soak testing of build '20160119' (see: https://wiki.hpdd.intel.com/pages/viewpage.action?title=Soak+Testing+on+Lola&spaceKey=Releases#SoakTestingonLola-20160119). MDTs have been formatted using ldisksfs, OSTs using zfs. DNE is enabled.

Upon booting the OSS nodes the first time after (re-)creation of the Lustre FS the ZFS pools aren't imported automatically.
After importing all zpools allocated for OSTs explicitly (i.e running zpool import -f <pool_name>), the pools will be constantly imported after each reboot. See attached logfiles

Comments

[Frank Heckes (Inactive)]

many thanks Nathaniel.

[Nathaniel Clark]

zpool import -f POOL will create /etc/zfs/zpool.cache which will cause the pools to be imported automatically each reboot.

[Frank Heckes (Inactive)]

The 'import' actions are logged in attached file 'cnventory-after-first-boot':

Script started on Fri 22 Jan 2016 03:02:04 AM PST
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# zpool list
no pools available
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# zpool lisESC[ESC[ESC[ESC[KimESC[Kport -f soaked-ost2
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# zpool import -f soaked-ostESC[KESC[K6
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# zpool import -f soaked-ostESC[K10
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# zpool import -f soaked-ost1ESC[K4
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
soaked-ost10  4.53T  24.4G  4.51T         -     0%     0%  1.00x  ONLINE  -
soaked-ost14  4.53T  14.6G  4.52T         -     0%     0%  1.00x  ONLINE  -
soaked-ost2   4.53T  17.4G  4.51T         -     0%     0%  1.00x  ONLINE  -
soaked-ost6   4.53T  10.2G  4.52T         -     0%     0%  1.00x  ONLINE  -
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# # Reboot on more time
ESC]0;root@lola-4:~ESC\[root@lola-4 ~]# exit
exit
Script done on Fri 22 Jan 2016 03:05:28 AM PST

Actually no error occurred. This was the same on all nodes I had to execute the command sequence.

[Nathaniel Clark]

Do you have a copy of the error you get when you import it the first time?

[Frank Heckes (Inactive)]

• I attached the script library. Indeed the essential command in line 258:

  on $host "zpool create -f $(poolname $target) -o cachefile=none $(dev $target)"
  

  create the zpool with no cachefile.
  This configuration option was decided to be used after some discussion about how-to import in HA set-up.

• What is the correct way to do it?
• No, errors occurred when explicitly importing the pool after the first boot after pool creation.
  And once this is done the pools are imported every the node is rebooted.

[Nathaniel Clark]

I don't see information on setup. A lot of setup scripts don't create a cache file on the hosts for zfs so automatic import doesn't happen -o cachefile=none. I don't see a link for the soak test scripts do you have the easily accessable? Where there any error when you tried to import them the first time?

[Peter Jones]

Nathaniel

Could you advise on this one?

Thanks

Peter