[LU-6452] lustre 2.7 zfs issue

• Type: Bug
• Resolution: Not a Bug
• Priority: Major
• Affects Version/s: Lustre 2.7.0
• Labels: zfs
• Environment: RHEL 6.5

Epic/Theme

zfs

• Severity: 3
• Project: Test Infrastructure
• Assignee: WC Triage
• Reporter: MzP

Description

Hi,
While trying to create a zfs mgs/mdt i get the following error.

mkfs.lustre --reformat --backfstype=zfs --fsname=lustre1 --mgs /dev/sdb

mkfs.lustre FATAL: Invalid filesystem name /dev/sdb

mkfs.lustre FATAL: unable to prepare backend (22)
mkfs.lustre: exiting with 22 (Invalid argument)

This was reported as an issue before for lustre 2.4 and was fixed then but may have come back again.

I have the following lustre and zfs packages installed
kernel-2.6.32-504.8.1.el6_lustre.x86_64
kernel-devel-2.6.32-504.8.1.el6_lustre.x86_64
kernel-firmware-2.6.32-504.8.1.el6_lustre.x86_64
kmod-spl-2.6.32-504.8.1.el6_lustre.x86_64-0.6.4-1.el6.x86_64
kmod-spl-devel-2.6.32-504.8.1.el6_lustre.x86_64-0.6.4-1.el6.x86_64
kmod-zfs-2.6.32-504.8.1.el6_lustre.x86_64-0.6.4-1.el6.x86_64
libzfs2-0.6.4-1.el6.x86_64
lustre-2.7.0-2.6.32_504.8.1.el6_lustre.x86_64.x86_64
lustre-iokit-2.7.0-2.6.32_504.8.1.el6_lustre.x86_64.x86_64
lustre-modules-2.7.0-2.6.32_504.8.1.el6_lustre.x86_64.x86_64
lustre-osd-zfs-2.7.0-2.6.32_504.8.1.el6_lustre.x86_64.x86_64
lustre-osd-zfs-mount-2.7.0-2.6.32_504.8.1.el6_lustre.x86_64.x86_64
lustre-tests-2.7.0-2.6.32_504.8.1.el6_lustre.x86_64.x86_64
zfs-0.6.4-1.el6.x86_64

/dev/sdb is a valid disk and exists on the machine

Comments

[Götz Waschk]

Binary packages of Lustre 2.7.0 depend on zfs/spl version 0.6.3-1.2 . This should be mentioned in the release notes.

[MzP]

Thanks, that worked. However it looks like I'm unable to load the osd_zfs modules.

FATAL: Error inserting osd_zfs (/lib/modules/2.6.32-504.8.1.el6_lustre.x86_64/extra/kernel/fs/lustre/osd_zfs.ko): Invalid argument

The module is present
rwxrr- 1 root root 527448 Mar 4 16:35 /lib/modules/2.6.32-504.8.1.el6_lustre.x86_64/extra/kernel/fs/lustre/osd_zfs.ko

From dmesg
osd_zfs: disagrees about version of symbol dsl_prop_register
osd_zfs: Unknown symbol dsl_prop_register (err -22)
osd_zfs: disagrees about version of symbol zap_cursor_serialize
osd_zfs: Unknown symbol zap_cursor_serialize (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_hold_write
osd_zfs: Unknown symbol dmu_tx_hold_write (err -22)
osd_zfs: disagrees about version of symbol dsl_prop_unregister
osd_zfs: Unknown symbol dsl_prop_unregister (err -22)
osd_zfs: disagrees about version of symbol sa_spill_rele
osd_zfs: Unknown symbol sa_spill_rele (err -22)
osd_zfs: disagrees about version of symbol zap_cursor_advance
osd_zfs: Unknown symbol zap_cursor_advance (err -22)
osd_zfs: disagrees about version of symbol dmu_prefetch
osd_zfs: Unknown symbol dmu_prefetch (err -22)
osd_zfs: disagrees about version of symbol dmu_read
osd_zfs: Unknown symbol dmu_read (err -22)
osd_zfs: disagrees about version of symbol zap_cursor_init_serialized
osd_zfs: Unknown symbol zap_cursor_init_serialized (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_assign
osd_zfs: Unknown symbol dmu_tx_assign (err -22)
osd_zfs: disagrees about version of symbol sa_handle_get
osd_zfs: Unknown symbol sa_handle_get (err -22)
osd_zfs: disagrees about version of symbol zap_cursor_fini
osd_zfs: Unknown symbol zap_cursor_fini (err -22)
osd_zfs: disagrees about version of symbol zrl_add
osd_zfs: Unknown symbol zrl_add (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_create
osd_zfs: Unknown symbol dmu_tx_create (err -22)
osd_zfs: disagrees about version of symbol sa_lookup
osd_zfs: Unknown symbol sa_lookup (err -22)
osd_zfs: disagrees about version of symbol dmu_object_alloc
osd_zfs: Unknown symbol dmu_object_alloc (err -22)
osd_zfs: disagrees about version of symbol dmu_object_next
osd_zfs: Unknown symbol dmu_object_next (err -22)
osd_zfs: disagrees about version of symbol dmu_object_free
osd_zfs: Unknown symbol dmu_object_free (err -22)
osd_zfs: disagrees about version of symbol dmu_objset_own
osd_zfs: Unknown symbol dmu_objset_own (err -22)
osd_zfs: disagrees about version of symbol __cv_broadcast
osd_zfs: Unknown symbol __cv_broadcast (err -22)
osd_zfs: disagrees about version of symbol zap_cursor_init
osd_zfs: Unknown symbol zap_cursor_init (err -22)
osd_zfs: disagrees about version of symbol dmu_write
osd_zfs: Unknown symbol dmu_write (err -22)
osd_zfs: disagrees about version of symbol sa_bulk_lookup
osd_zfs: Unknown symbol sa_bulk_lookup (err -22)
osd_zfs: disagrees about version of symbol dmu_return_arcbuf
osd_zfs: Unknown symbol dmu_return_arcbuf (err -22)
osd_zfs: disagrees about version of symbol dmu_assign_arcbuf
osd_zfs: Unknown symbol dmu_assign_arcbuf (err -22)
osd_zfs: disagrees about version of symbol sa_object_size
osd_zfs: Unknown symbol sa_object_size (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_callback_register
osd_zfs: Unknown symbol dmu_tx_callback_register (err -22)
osd_zfs: disagrees about version of symbol sa_setup
osd_zfs: Unknown symbol sa_setup (err -22)
osd_zfs: disagrees about version of symbol sa_update
osd_zfs: Unknown symbol sa_update (err -22)
osd_zfs: disagrees about version of symbol dmu_objset_disown
osd_zfs: Unknown symbol dmu_objset_disown (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_hold_sa_create
osd_zfs: Unknown symbol dmu_tx_hold_sa_create (err -22)
osd_zfs: disagrees about version of symbol dmu_objset_ds
osd_zfs: Unknown symbol dmu_objset_ds (err -22)
osd_zfs: disagrees about version of symbol dmu_objset_spa
osd_zfs: Unknown symbol dmu_objset_spa (err -22)
osd_zfs: disagrees about version of symbol spa_get_dsl
osd_zfs: Unknown symbol spa_get_dsl (err -22)
osd_zfs: Unknown symbol sa_spill_free (err 0)
osd_zfs: disagrees about version of symbol dmu_objset_userused_enabled
osd_zfs: Unknown symbol dmu_objset_userused_enabled (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_hold_zap
osd_zfs: Unknown symbol dmu_tx_hold_zap (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_get_txg
osd_zfs: Unknown symbol dmu_tx_get_txg (err -22)
osd_zfs: disagrees about version of symbol dmu_request_arcbuf
osd_zfs: Unknown symbol dmu_request_arcbuf (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_commit
osd_zfs: Unknown symbol dmu_tx_commit (err -22)
osd_zfs: disagrees about version of symbol txg_wait_callbacks
osd_zfs: Unknown symbol txg_wait_callbacks (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_hold_sa
osd_zfs: Unknown symbol dmu_tx_hold_sa (err -22)
osd_zfs: disagrees about version of symbol dsl_pool_config_enter
osd_zfs: Unknown symbol dsl_pool_config_enter (err -22)
osd_zfs: disagrees about version of symbol dbuf_hold_impl
osd_zfs: Unknown symbol dbuf_hold_impl (err -22)
osd_zfs: Unknown symbol __gethrestime (err 0)
osd_zfs: disagrees about version of symbol dmu_objset_pool
osd_zfs: Unknown symbol dmu_objset_pool (err -22)
osd_zfs: disagrees about version of symbol dmu_free_range
osd_zfs: Unknown symbol dmu_free_range (err -22)
osd_zfs: disagrees about version of symbol sa_size
osd_zfs: Unknown symbol sa_size (err -22)
osd_zfs: disagrees about version of symbol sa_buf_hold
osd_zfs: Unknown symbol sa_buf_hold (err -22)
osd_zfs: disagrees about version of symbol sa_replace_all_by_template
osd_zfs: Unknown symbol sa_replace_all_by_template (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_hold_bonus
osd_zfs: Unknown symbol dmu_tx_hold_bonus (err -22)
osd_zfs: disagrees about version of symbol spa_freeze
osd_zfs: Unknown symbol spa_freeze (err -22)
osd_zfs: disagrees about version of symbol zap_cursor_retrieve
osd_zfs: Unknown symbol zap_cursor_retrieve (err -22)
osd_zfs: disagrees about version of symbol zrl_remove
osd_zfs: Unknown symbol zrl_remove (err -22)
osd_zfs: disagrees about version of symbol txg_wait_synced
osd_zfs: Unknown symbol txg_wait_synced (err -22)
osd_zfs: Unknown symbol sa_spill_alloc (err 0)
osd_zfs: disagrees about version of symbol dmu_object_info_from_db
osd_zfs: Unknown symbol dmu_object_info_from_db (err -22)
osd_zfs: disagrees about version of symbol dsl_pool_config_exit
osd_zfs: Unknown symbol dsl_pool_config_exit (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_abort
osd_zfs: Unknown symbol dmu_tx_abort (err -22)
osd_zfs: disagrees about version of symbol sa_handle_destroy
osd_zfs: Unknown symbol sa_handle_destroy (err -22)
osd_zfs: disagrees about version of symbol dmu_objset_space
osd_zfs: Unknown symbol dmu_objset_space (err -22)
osd_zfs: disagrees about version of symbol dbuf_rele
osd_zfs: Unknown symbol dbuf_rele (err -22)
osd_zfs: disagrees about version of symbol dmu_object_set_blocksize
osd_zfs: Unknown symbol dmu_object_set_blocksize (err -22)
osd_zfs: disagrees about version of symbol sa_bulk_update
osd_zfs: Unknown symbol sa_bulk_update (err -22)
osd_zfs: disagrees about version of symbol dmu_tx_hold_free
osd_zfs: Unknown symbol dmu_tx_hold_free (err -22)

[Andreas Dilger]

From the mkfs.lustre(8) man page:

mkfs.lustre(8)              configuration utilities             mkfs.lustre(8)

NAME
       mkfs.lustre - format a disk for a Lustre service

SYNOPSIS
       mkfs.lustre {--ost|--mdt|--mgs} --backfstype=zfs --fsname=<name>
           [options] <pool_name>/<dataset_name> [<zpool_specification>]
:
:
OPTIONS
       --backfstype=fstype
              Force a particular format for the backing fs (ldiskfs, zfs).
              zpool_specification = [[<vdev_type>] <device> [<device> ...] [<vdev_type>] ...]
              vdev_type = {mirror, raidz, raidz2, raidz3, cache}
              device = { Linux block device }

              If no vdev_type is given, then the devices are used in a round-
              robin (striped) manner. See zpool(8) for more details.

[Bruno Faccini (Inactive)]

You need to use/pass a device with ZFS syntax (zpool/dataset).