[LU-9370] Lustre 2.9 + zfs 0.7 + draid = OSS hangup

Thu, 21 Dec 2017 18:58:47 +0000
I'm trying to build a draid based OST for lustre.
Initially created as [https://github.com/thegreatgazoo/zfs/issues/2|thegreatgazoo/zfs issue].
Generic lustre MGS/MDT is up'n'running.
Fresh VM (4 CPU, 4G RAM) with CentOS 7 "minimal" install and 18 scsi disks (images).
Perform yum -y update ; reboot, then run setup-node.sh NODE from workstation.
Ssh to the NODE and run ./mkzpool.sh:
[root@node26 ~]# ./mkzpool.sh 
+ zpool list
+ grep -w 'no pools available'
+ zpool destroy oss3pool
+ zpool list
+ grep -w 'no pools available'
no pools available
+ '[' -f 17.nvl ']'
+ draidcfg -r 17.nvl
dRAID1 vdev of 17 child drives: 3 x (4 data + 1 parity) and 2 distributed spare
Using 32 base permutations
  15, 2, 8, 7,10, 5, 4,16, 1,13,14, 9,11,12, 3, 6, 0,
   5,15,14, 9, 0,11,13, 4, 3,12, 8,10, 7, 1, 6, 2,16,
  10,11,14, 5,15, 2,13, 6, 1, 3, 4, 7,12,16, 9, 0, 8,
  13, 2,12,14, 8, 0, 7, 4, 9,15,11, 6, 3,16, 1, 5,10,
  13, 5, 2,16, 6, 0, 4, 8,10, 1, 3,14, 9,11,12, 7,15,
   8,12, 3,14, 0, 4,16, 6, 2,11, 1, 7, 9,15,13, 5,10,
  16,14, 2, 9, 7, 4,11, 0, 6,12,10, 8, 1,13,15, 5, 3,
   5,16, 6, 1,10,15,11, 3, 8,14, 2,12, 0, 7, 9, 4,13,
   4,12, 8,10,14, 9, 6,11,15, 0, 3,13, 7, 2, 5,16, 1,
  10,14,16,11,12, 2, 5, 3, 4, 7, 0, 1, 6, 9,13, 8,15,
   2, 1,11,15,16, 6,12, 3,10,13, 8, 5, 4, 0, 7, 9,14,
  15,14, 1, 5,16, 2,12, 8, 9, 6,11,10, 3, 0, 7, 4,13,
   1, 5,10, 9, 2, 8, 4,16, 7,11, 3,12, 6,14, 0,13,15,
   3, 7,16,10,13, 2, 6, 8,14,15,12,11, 0, 9, 1, 4, 5,
  15, 2,14, 8, 5,16, 3,13, 4, 1, 9,12,10, 0, 6, 7,11,
  14,12,11,15,16,10, 2, 9, 8, 4, 3, 1,13, 5, 7, 0, 6,
   7,13, 2,11,14, 0, 1, 8, 9,10,16, 4, 6,12, 5, 3,15,
  16, 1,11, 4, 3, 9, 6,13, 5, 7,10,15,14,12, 2, 0, 8,
   0, 5, 2,10,16,12, 6, 3,11,14, 1, 9, 7,15, 4, 8,13,
   8,13,11, 4,10, 6, 7,16, 5,12, 9,14, 2, 3, 0,15, 1,
   9, 6,12,16, 4, 7, 3, 0, 2,15,13, 8,11,14, 5,10, 1,
   8,12, 0, 6,15, 7, 4,13,14,10, 1, 9, 5, 3,11, 2,16,
   5,15, 9,10,16, 6,11, 0, 7,13, 8,14, 3, 4, 1,12, 2,
  15,14, 2, 9, 4,11, 7, 1, 6,10, 5, 0, 8,12,13,16, 3,
  15,16, 0,10, 3,12,11, 7, 1, 8, 6,13, 4, 5, 9, 2,14,
  15, 4, 7,13,14, 2, 9,10,16, 1,11,12, 8, 0, 3, 5, 6,
  15, 8,13, 0, 4, 7, 3,14, 5,12, 2, 9,10,11, 6,16, 1,
   0, 7, 5, 3, 1,14,16, 4, 2,15,12, 8,10, 6, 9,11,13,
   7, 6, 0,15,16,11, 8, 1, 5,12,13,14,10, 9, 3, 2, 4,
  14,16,10, 6, 4,13, 3, 1,15,12,11, 8, 9, 5, 0, 7, 2,
   9, 3, 5,15,10,11, 8, 7, 2,14, 6,13, 0, 4, 1,12,16,
   4, 6, 7,14, 5, 3,12, 1,13, 9,16, 2, 0,10, 8,11,15,
+ zpool create -f oss3pool draid1 cfg=17.nvl /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo /dev/sdp /dev/sdq /dev/sdr
+ zpool list
NAME       SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
oss3pool  14,7G   612K  14,7G         -     0%     0%  1.00x  ONLINE  -
+ zpool status
  pool: oss3pool
 state: ONLINE
  scan: none requested
config:

	NAME            STATE     READ WRITE CKSUM
	oss3pool        ONLINE       0     0     0
	  draid1-0      ONLINE       0     0     0
	    sdb         ONLINE       0     0     0
	    sdc         ONLINE       0     0     0
	    sdd         ONLINE       0     0     0
	    sde         ONLINE       0     0     0
	    sdf         ONLINE       0     0     0
	    sdg         ONLINE       0     0     0
	    sdh         ONLINE       0     0     0
	    sdi         ONLINE       0     0     0
	    sdj         ONLINE       0     0     0
	    sdk         ONLINE       0     0     0
	    sdl         ONLINE       0     0     0
	    sdm         ONLINE       0     0     0
	    sdn         ONLINE       0     0     0
	    sdo         ONLINE       0     0     0
	    sdp         ONLINE       0     0     0
	    sdq         ONLINE       0     0     0
	    sdr         ONLINE       0     0     0
	spares
	  $draid1-0-s0  AVAIL   
	  $draid1-0-s1  AVAIL   

errors: No known data errors
+ grep oss3pool
+ mount
oss3pool on /oss3pool type zfs (rw,xattr,noacl)
+ mkfs.lustre --reformat --ost --backfstype=zfs --fsname=ZFS01 --index=3 --mgsnode=mgs@tcp0 oss3pool/ZFS01

   Permanent disk data:
Target:     ZFS01:OST0003
Index:      3
Lustre FS:  ZFS01
Mount type: zfs
Flags:      0x62
              (OST first_time update )
Persistent mount opts: 
Parameters: mgsnode=172.17.32.220@tcp

mkfs_cmd = zfs create -o canmount=off -o xattr=sa oss3pool/ZFS01
Writing oss3pool/ZFS01 properties
  lustre:version=1
  lustre:flags=98
  lustre:index=3
  lustre:fsname=ZFS01
  lustre:svname=ZFS01:OST0003
  lustre:mgsnode=172.17.32.220@tcp
+ '[' -d /lustre/ZFS01/. ']'
+ mount -v -t lustre oss3pool/ZFS01 /lustre/ZFS01
arg[0] = /sbin/mount.lustre
arg[1] = -v
arg[2] = -o
arg[3] = rw
arg[4] = oss3pool/ZFS01
arg[5] = /lustre/ZFS01
source = oss3pool/ZFS01 (oss3pool/ZFS01), target = /lustre/ZFS01
options = rw
checking for existing Lustre data: found
Writing oss3pool/ZFS01 properties
  lustre:version=1
  lustre:flags=34
  lustre:index=3
  lustre:fsname=ZFS01
  lustre:svname=ZFS01:OST0003
  lustre:mgsnode=172.17.32.220@tcp
mounting device oss3pool/ZFS01 at /lustre/ZFS01, flags=0x1000000 options=osd=osd-zfs,,mgsnode=172.17.32.220@tcp,virgin,update,param=mgsnode=172.17.32.220@tcp,svname=ZFS01-OST0003,device=oss3pool/ZFS01
mount.lustre: mount oss3pool/ZFS01 at /lustre/ZFS01 failed: Address already in use retries left: 0
mount.lustre: mount oss3pool/ZFS01 at /lustre/ZFS01 failed: Address already in use
The target service's index is already in use. (oss3pool/ZFS01)
[root@node26 ~]# mount -v -t lustre oss3pool/ZFS01 /lustre/ZFS01
arg[0] = /sbin/mount.lustre
arg[1] = -v
arg[2] = -o
arg[3] = rw
arg[4] = oss3pool/ZFS01
arg[5] = /lustre/ZFS01
source = oss3pool/ZFS01 (oss3pool/ZFS01), target = /lustre/ZFS01
options = rw
checking for existing Lustre data: found
mounting device oss3pool/ZFS01 at /lustre/ZFS01, flags=0x1000000 options=osd=osd-zfs,,mgsnode=172.17.32.220@tcp,virgin,param=mgsnode=172.17.32.220@tcp,svname=ZFS01-OST0003,device=oss3pool/ZFS01
Whamcloud Community JIRA

[LU-9370] Lustre 2.9 + zfs 0.7 + draid = OSS hangup
