Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
Lustre 2.10.0
-
3
-
9223372036854775807
Description
Under zfs 0.7 zpool create fails with this response, anything has changed in version 0.7 ?
zpool create -f -o ashift=12 ost raidz2 35000c5008355e2c3 35000c5008355cedb 35000c500837ece67 35000c5008355ddcf 35000c500837f23cf 35000c500837f2b93 35000c5008355bb67 35000c5008355bbdf 35000c5008355da8b 35000c500837f249f 35000c5008355c33b
cannot create 'ost': invalid argument for this pool operation
rpm -qa "zpool"
libzpool2-0.7.0-1.el7_3.x86_64
dkms status
lustre, 2.10.0, 3.10.0-514.21.1.el7_lustre.x86_64, x86_64: installed
spl, 0.7.0, 3.10.0-514.21.1.el7_lustre.x86_64, x86_64: installed
zfs, 0.7.0, 3.10.0-514.21.1.el7_lustre.x86_64, x86_64: installed
thanks,
Abe
Attachments
Activity
Hi Nathaniel,
The problem went away after i rebooted the server, however still like to know
the root cause..
I have uploaded the sosreport, pls take a look and let me know your findings.
[root@OSS1-A212 tmp]# ncftp ftp.hpdd.intel.com
NcFTP 3.2.5 (Feb 02, 2011) by Mike Gleason (http://www.NcFTP.com/contact/).
Copyright (c) 1992-2011 by Mike Gleason.
All rights reserved.
Connecting to ftp.hpdd.intel.com...
Welcome to the FTP service on ftp-1.onyx.hpdd.intel.com. Please be nice.
Logging in...
Login successful.
Logged in to ftp.hpdd.intel.com.
ncftp / > cd uploads
Directory successfully changed.
ncftp /uploads > mkdir LU-9852
ncftp /uploads > cd LU-9852
Directory successfully changed.
ncftp /uploads/LU-9852 > ls
ncftp /uploads/LU-9852 > put sosreport-AbeAsraoui-20170811194052.tar.xz
sosreport-AbeAsraoui-20170811194052.tar.xz: 102.83 MB 2.43 MB/s
Could not preserve times for sosreport-AbeAsraoui-20170811194052.tar.xz: UTIME failed.
ncftp /uploads/LU-9852 > ls -l sos*
rw-rr- 1 99 50 107827736 Aug 22 2017 sosreport-AbeAsraoui-20170811194052.tar.xz
thanks,
Abe
Here are the following instructions for ftp upload:
Sometimes the diagnostic data collected as part of Lustre troubleshooting is too large to be attached to a JIRA ticket. For these cases, HPDD provides an anonymous write-only FTP upload service. In order to use this service, you'll need an FTP client (e.g. ncftp, ftp, etc.) and a JIRA issue. Use the 'uploads' directory and create a new subdirectory using your Jira issue as a name.
In the following example, there are three debug logs in a single directory and the JIRA issue LU-4242 has been created. After completing the upload, please update the relevant issue with a note mentioning the upload, so that our engineers know where to find your logs.
$ ls -lh total 333M -rw-r--r-- 1 mjmac mjmac 98M Feb 23 17:36 mds-debug -rw-r--r-- 1 mjmac mjmac 118M Feb 23 17:37 oss-00-debug -rw-r--r-- 1 mjmac mjmac 118M Feb 23 17:37 oss-01-debug $ ncftp ftp.hpdd.intel.com NcFTP 3.2.2 (Sep 04, 2008) by Mike Gleason (http://www.NcFTP.com/contact/). Connecting to 99.96.190.235... (vsFTPd 2.2.2) Logging in... Login successful. Logged in to ftp.hpdd.intel.com. ncftp / > cd uploads Directory successfully changed. ncftp /uploads > mkdir LU-4242 ncftp /uploads > cd LU-4242 Directory successfully changed. ncftp /uploads/LU-4242 > put * mds-debug: 97.66 MB 11.22 MB/s oss-00-debug: 117.19 MB 11.16 MB/s oss-01-debug: 117.48 MB 11.18 MB/s ncftp /uploads/LU-4242 >
Please note that this is a WRITE-ONLY FTP service, so you will not be able to see (with ls) the files or directories you've created, nor will you (or anyone other than HPDD staff) be able to see or read them.
Hi Nathaniel,
is there a tftp server i can ftp the the compressed tar file to ?
thanks,
Abe
not able to upload the sos report, is there a tftp server or an email where i can upload this ?
File "sosreport-AbeAsraoui-20170811194052.tar.xz" was not uploaded
File exceeds maximum size limit set by the administrator (100.00 MB).
thanks,
Abe
sorry not familiar with this sosreport, how do i run it and how to download this tool..
thanks,
Abe
it looks like it is mapped:
e.g.
zpool create -f -o ashift=12 ost1 raidz2 35000c5008355e01f 35000c5008355ce6b 35000c500837f714b 35000c5008355dc6f 35000c50083551f3b 35000c500837eccbb 35000c5008355229f 35000c5008357281b 35000c50083571977 35000c50083571ceb 35000c500835550c7
lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c5008355e01f -> ../dm-4
lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c5008355ce6b -> ../dm-5
lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c500837f714b -> ../dm-6
lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c5008355dc6f -> ../dm-7
lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c50083551f3b -> ../dm-8
lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c500837eccbb -> ../dm-9
lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c5008355229f -> ../dm-10
lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c5008357281b -> ../dm-11
lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c50083571977 -> ../dm-12
lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c50083571ceb -> ../dm-13
lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c500835550c7 -> ../dm-14
The mpath's listed in the create, aren't listed in /dev/mapper/. Maybe the LUs aren't mapped to this server?
here are the paths for the drives on this oss server, the strange thing is that i was able to add
the drives on a different oss server but not this one:
lrwxrwxrwx 1 root root 8 Aug 9 20:33 /dev/mapper/35000c500837f2a77 -> ../dm-92
lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c500837f23c7 -> ../dm-3
lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c5008355e01f -> ../dm-4
lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c5008355ce6b -> ../dm-5
lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c500837f714b -> ../dm-6
lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c5008355dc6f -> ../dm-7
lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c50083551f3b -> ../dm-8
lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c500837eccbb -> ../dm-9
lrwxrwxrwx 1 root root 8 Aug 10 18:37 /dev/mapper/35000c5008355229f -> ../dm-10
lrwxrwxrwx 1 root root 8 Aug 10 18:37 /dev/mapper/35000c5008357281b -> ../dm-11
lrwxrwxrwx 1 root root 8 Aug 10 18:37 /dev/mapper/35000c50083571977 -> ../dm-12
created the zpools on the other server:
OSS1-A210 obd]# zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
mgs 3.62T 612K 3.62T - 0% 0% 1.00x ONLINE -
ost5 40T 11.1M 40.0T - 0% 0% 1.00x ONLINE -
ost6 40T 11.0M 40.0T - 0% 0% 1.00x ONLINE -
ost7 40T 11.1M 40.0T - 0% 0% 1.00x ONLINE -
ost8 40T 11.1M 40.0T - 0% 0% 1.00x ONLINE -
zpool status
pool: mgs
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
mgs ONLINE 0 0 0
35000c500837f2a77 ONLINE 0 0 0
errors: No known data errors
pool: ost5
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
ost5 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
35000c5008355c23f ONLINE 0 0 0
35000c500837f23f7 ONLINE 0 0 0
35000c5008355cd9f ONLINE 0 0 0
35000c500835529ef ONLINE 0 0 0
35000c50083570f43 ONLINE 0 0 0
35000c500837ecee3 ONLINE 0 0 0
35000c5008355e3ef ONLINE 0 0 0
35000c500835709d3 ONLINE 0 0 0
35000c5008355c19b ONLINE 0 0 0
35000c5008355bda3 ONLINE 0 0 0
35000c5008355b603 ONLINE 0 0 0
errors: No known data errors
pool: ost6
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
ost6 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
35000c5008355dadb ONLINE 0 0 0
35000c50083554af7 ONLINE 0 0 0
35000c5008355cd5b ONLINE 0 0 0
35000c5008355b7cf ONLINE 0 0 0
35000c5008355c0f3 ONLINE 0 0 0
35000c5008355d513 ONLINE 0 0 0
35000c500837f24c7 ONLINE 0 0 0
35000c5008355da9f ONLINE 0 0 0
35000c500837f2a37 ONLINE 0 0 0
35000c50083555217 ONLINE 0 0 0
35000c500837f2637 ONLINE 0 0 0
errors: No known data errors
pool: ost7
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
ost7 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
35000c50083571913 ONLINE 0 0 0
35000c500837f24f7 ONLINE 0 0 0
35000c5008355dd6f ONLINE 0 0 0
35000c5008355dcfb ONLINE 0 0 0
35000c5008355d73f ONLINE 0 0 0
35000c50083553b8f ONLINE 0 0 0
35000c50083571d7b ONLINE 0 0 0
35000c5008355ca93 ONLINE 0 0 0
35000c50083570b87 ONLINE 0 0 0
35000c50083555653 ONLINE 0 0 0
35000c5008355bba7 ONLINE 0 0 0
errors: No known data errors
pool: ost8
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
ost8 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
35000c50083554003 ONLINE 0 0 0
35000c50083571937 ONLINE 0 0 0
35000c5008355bd23 ONLINE 0 0 0
35000c500837f2153 ONLINE 0 0 0
35000c5008355d007 ONLINE 0 0 0
35000c50083553007 ONLINE 0 0 0
35000c50083572bb3 ONLINE 0 0 0
35000c5008355d763 ONLINE 0 0 0
35000c500837f11ef ONLINE 0 0 0
35000c5008355225b ONLINE 0 0 0
35000c5008355e177 ONLINE 0 0 0
errors: No known data errors
[root@OSS1-A210 obd]#
thanks,
Abe
Can you double check that those devices exist?
The default search path is:
"/dev/disk/by-vdev" "/dev/mapper" "/dev/disk/by-partlabel" "/dev/disk/by-partuuid" "/dev/disk/by-label" "/dev/disk/by-uuid" "/dev/disk/by-id" "/dev/disk/by-path" "/dev"
Looking at the log, it looks like you had ZFS 0.6.5.9 installed then upgraded to 0.7.0, but the old modules weren't unloaded prior to upgrade, so I think that is where the issue lay, and why a reboot fixed it.