https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9852 Lustre <p>Under zfs 0.7 zpool create fails with this response, anything has changed in version 0.7 ?</p> <p>zpool create -f -o ashift=12 ost raidz2 35000c5008355e2c3 35000c5008355cedb 35000c500837ece67 35000c5008355ddcf 35000c500837f23cf 35000c500837f2b93 35000c5008355bb67 35000c5008355bbdf 35000c5008355da8b 35000c500837f249f 35000c5008355c33b</p> <p>cannot create 'ost': invalid argument for this pool operation<br/> rpm -qa "zpool"<br/> libzpool2-0.7.0-1.el7_3.x86_64</p> <p> dkms status<br/> lustre, 2.10.0, 3.10.0-514.21.1.el7_lustre.x86_64, x86_64: installed<br/> spl, 0.7.0, 3.10.0-514.21.1.el7_lustre.x86_64, x86_64: installed<br/> zfs, 0.7.0, 3.10.0-514.21.1.el7_lustre.x86_64, x86_64: installed</p> <p>thanks,<br/> Abe</p> LU-9852

zfs pool creation fails in version 0.7.0

Bug Major Resolved Not a Bug Nathaniel Clark Abe zfs Wed, 9 Aug 2017 23:52:07 +0000 Wed, 1 Nov 2017 17:28:50 +0000 Thu, 31 Aug 2017 04:41:47 +0000 Lustre 2.10.0 0 3 <p>Nathaniel</p> <p>Can you please advise?</p> <p>Thanks</p> <p>Peter</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=abea%40supermicro.com" class="user-hover" rel="abea@supermicro.com">abea@supermicro.com</a>,</p> <p>Can you double check that those devices exist?<br/> The default search path is:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> "/dev/disk/by-vdev" "/dev/mapper" "/dev/disk/by-partlabel" "/dev/disk/by-partuuid" "/dev/disk/by-label" "/dev/disk/by-uuid" "/dev/disk/by-id" "/dev/disk/by-path" "/dev" </pre> </div></div> <p>here are the paths for the drives on this oss server, the strange thing is that i was able to add <br/> the drives on a different oss server but not this one:</p> <p>lrwxrwxrwx 1 root root 8 Aug 9 20:33 /dev/mapper/35000c500837f2a77 -&gt; ../dm-92<br/> lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c500837f23c7 -&gt; ../dm-3<br/> lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c5008355e01f -&gt; ../dm-4<br/> lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c5008355ce6b -&gt; ../dm-5<br/> lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c500837f714b -&gt; ../dm-6<br/> lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c5008355dc6f -&gt; ../dm-7<br/> lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c50083551f3b -&gt; ../dm-8<br/> lrwxrwxrwx 1 root root 7 Aug 10 18:37 /dev/mapper/35000c500837eccbb -&gt; ../dm-9<br/> lrwxrwxrwx 1 root root 8 Aug 10 18:37 /dev/mapper/35000c5008355229f -&gt; ../dm-10<br/> lrwxrwxrwx 1 root root 8 Aug 10 18:37 /dev/mapper/35000c5008357281b -&gt; ../dm-11<br/> lrwxrwxrwx 1 root root 8 Aug 10 18:37 /dev/mapper/35000c50083571977 -&gt; ../dm-12</p> <p>created the zpools on the other server:</p> <p>OSS1-A210 obd]# zpool list<br/> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT<br/> mgs 3.62T 612K 3.62T - 0% 0% 1.00x ONLINE -<br/> ost5 40T 11.1M 40.0T - 0% 0% 1.00x ONLINE -<br/> ost6 40T 11.0M 40.0T - 0% 0% 1.00x ONLINE -<br/> ost7 40T 11.1M 40.0T - 0% 0% 1.00x ONLINE -<br/> ost8 40T 11.1M 40.0T - 0% 0% 1.00x ONLINE -</p> <p>zpool status<br/> pool: mgs<br/> state: ONLINE<br/> scan: none requested<br/> config:</p> <p> NAME STATE READ WRITE CKSUM<br/> mgs ONLINE 0 0 0<br/> 35000c500837f2a77 ONLINE 0 0 0</p> <p>errors: No known data errors</p> <p> pool: ost5<br/> state: ONLINE<br/> scan: none requested<br/> config:</p> <p> NAME STATE READ WRITE CKSUM<br/> ost5 ONLINE 0 0 0<br/> raidz2-0 ONLINE 0 0 0<br/> 35000c5008355c23f ONLINE 0 0 0<br/> 35000c500837f23f7 ONLINE 0 0 0<br/> 35000c5008355cd9f ONLINE 0 0 0<br/> 35000c500835529ef ONLINE 0 0 0<br/> 35000c50083570f43 ONLINE 0 0 0<br/> 35000c500837ecee3 ONLINE 0 0 0<br/> 35000c5008355e3ef ONLINE 0 0 0<br/> 35000c500835709d3 ONLINE 0 0 0<br/> 35000c5008355c19b ONLINE 0 0 0<br/> 35000c5008355bda3 ONLINE 0 0 0<br/> 35000c5008355b603 ONLINE 0 0 0</p> <p>errors: No known data errors</p> <p> pool: ost6<br/> state: ONLINE<br/> scan: none requested<br/> config:</p> <p> NAME STATE READ WRITE CKSUM<br/> ost6 ONLINE 0 0 0<br/> raidz2-0 ONLINE 0 0 0<br/> 35000c5008355dadb ONLINE 0 0 0<br/> 35000c50083554af7 ONLINE 0 0 0<br/> 35000c5008355cd5b ONLINE 0 0 0<br/> 35000c5008355b7cf ONLINE 0 0 0<br/> 35000c5008355c0f3 ONLINE 0 0 0<br/> 35000c5008355d513 ONLINE 0 0 0<br/> 35000c500837f24c7 ONLINE 0 0 0<br/> 35000c5008355da9f ONLINE 0 0 0<br/> 35000c500837f2a37 ONLINE 0 0 0<br/> 35000c50083555217 ONLINE 0 0 0<br/> 35000c500837f2637 ONLINE 0 0 0</p> <p>errors: No known data errors</p> <p> pool: ost7<br/> state: ONLINE<br/> scan: none requested<br/> config:</p> <p> NAME STATE READ WRITE CKSUM<br/> ost7 ONLINE 0 0 0<br/> raidz2-0 ONLINE 0 0 0<br/> 35000c50083571913 ONLINE 0 0 0<br/> 35000c500837f24f7 ONLINE 0 0 0<br/> 35000c5008355dd6f ONLINE 0 0 0<br/> 35000c5008355dcfb ONLINE 0 0 0<br/> 35000c5008355d73f ONLINE 0 0 0<br/> 35000c50083553b8f ONLINE 0 0 0<br/> 35000c50083571d7b ONLINE 0 0 0<br/> 35000c5008355ca93 ONLINE 0 0 0<br/> 35000c50083570b87 ONLINE 0 0 0<br/> 35000c50083555653 ONLINE 0 0 0<br/> 35000c5008355bba7 ONLINE 0 0 0</p> <p>errors: No known data errors</p> <p> pool: ost8<br/> state: ONLINE<br/> scan: none requested<br/> config:</p> <p> NAME STATE READ WRITE CKSUM<br/> ost8 ONLINE 0 0 0<br/> raidz2-0 ONLINE 0 0 0<br/> 35000c50083554003 ONLINE 0 0 0<br/> 35000c50083571937 ONLINE 0 0 0<br/> 35000c5008355bd23 ONLINE 0 0 0<br/> 35000c500837f2153 ONLINE 0 0 0<br/> 35000c5008355d007 ONLINE 0 0 0<br/> 35000c50083553007 ONLINE 0 0 0<br/> 35000c50083572bb3 ONLINE 0 0 0<br/> 35000c5008355d763 ONLINE 0 0 0<br/> 35000c500837f11ef ONLINE 0 0 0<br/> 35000c5008355225b ONLINE 0 0 0<br/> 35000c5008355e177 ONLINE 0 0 0</p> <p>errors: No known data errors<br/> <span class="error">&#91;root@OSS1-A210 obd&#93;</span>#</p> <p>thanks,<br/> Abe</p> <p>The mpath's listed in the create, aren't listed in /dev/mapper/. Maybe the LUs aren't mapped to this server?</p> <p>it looks like it is mapped:<br/> e.g.<br/> zpool create -f -o ashift=12 ost1 raidz2 35000c5008355e01f 35000c5008355ce6b 35000c500837f714b 35000c5008355dc6f 35000c50083551f3b 35000c500837eccbb 35000c5008355229f 35000c5008357281b 35000c50083571977 35000c50083571ceb 35000c500835550c7</p> <p>lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c5008355e01f -&gt; ../dm-4<br/> lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c5008355ce6b -&gt; ../dm-5<br/> lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c500837f714b -&gt; ../dm-6<br/> lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c5008355dc6f -&gt; ../dm-7<br/> lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c50083551f3b -&gt; ../dm-8<br/> lrwxrwxrwx 1 root root 7 Aug 11 13:42 /dev/mapper/35000c500837eccbb -&gt; ../dm-9<br/> lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c5008355229f -&gt; ../dm-10<br/> lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c5008357281b -&gt; ../dm-11<br/> lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c50083571977 -&gt; ../dm-12<br/> lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c50083571ceb -&gt; ../dm-13<br/> lrwxrwxrwx 1 root root 8 Aug 11 13:42 /dev/mapper/35000c500835550c7 -&gt; ../dm-14</p> <p>Can you attach an sosreport from the affected OSS?</p> <p>sorry not familiar with this sosreport, how do i run it and how to download this tool..</p> <p>thanks,<br/> Abe</p> <p>not able to upload the sos report, is there a tftp server or an email where i can upload this ?</p> <p>File "sosreport-AbeAsraoui-20170811194052.tar.xz" was not uploaded<br/> File exceeds maximum size limit set by the administrator (100.00 MB).</p> <p>thanks,<br/> Abe</p> <p>Hi Nathaniel,</p> <p>is there a tftp server i can ftp the the compressed tar file to ?</p> <p>thanks,<br/> Abe</p> <p>Here are the following instructions for ftp upload:</p> <p>Sometimes the diagnostic data collected as part of Lustre troubleshooting is too large to be attached to a JIRA ticket. For these cases, HPDD provides an anonymous write-only FTP upload service. In order to use this service, you'll need an FTP client (e.g. ncftp, ftp, etc.) and a JIRA issue. Use the 'uploads' directory and create a new subdirectory using your Jira issue as a name. <br/> In the following example, there are three debug logs in a single directory and the JIRA issue <a href="https://jira.whamcloud.com/browse/LU-4242" title="mdt_open.c:1685:mdt_reint_open()) LBUG" class="issue-link" data-issue-key="LU-4242"><del>LU-4242</del></a> has been created. After completing the upload, please update the relevant issue with a note mentioning the upload, so that our engineers know where to find your logs.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>$ ls -lh total 333M -rw-r--r-- 1 mjmac mjmac 98M Feb 23 17:36 mds-debug -rw-r--r-- 1 mjmac mjmac 118M Feb 23 17:37 oss-00-debug -rw-r--r-- 1 mjmac mjmac 118M Feb 23 17:37 oss-01-debug $ ncftp ftp.hpdd.intel.com NcFTP 3.2.2 (Sep 04, 2008) by Mike Gleason (http://www.NcFTP.com/contact/). Connecting to 99.96.190.235... (vsFTPd 2.2.2) Logging in... Login successful. Logged in to ftp.hpdd.intel.com. ncftp / &gt; cd uploads Directory successfully changed. ncftp /uploads &gt; mkdir LU-4242 ncftp /uploads &gt; cd LU-4242 Directory successfully changed. ncftp /uploads/LU-4242 &gt; put * mds-debug: 97.66 MB 11.22 MB/s oss-00-debug: 117.19 MB 11.16 MB/s oss-01-debug: 117.48 MB 11.18 MB/s ncftp /uploads/LU-4242 &gt; </pre> </div></div> <p>Please note that this is a WRITE-ONLY FTP service, so you will not be able to see (with ls) the files or directories you've created, nor will you (or anyone other than HPDD staff) be able to see or read them.</p> <p>Hi Nathaniel,</p> <p>The problem went away after i rebooted the server, however still like to know<br/> the root cause..<br/> I have uploaded the sosreport, pls take a look and let me know your findings.</p> <p><span class="error">&#91;root@OSS1-A212 tmp&#93;</span># ncftp ftp.hpdd.intel.com<br/> NcFTP 3.2.5 (Feb 02, 2011) by Mike Gleason (<a href="http://www.NcFTP.com/contact/" class="external-link" target="_blank" rel="nofollow noopener">http://www.NcFTP.com/contact/</a>).</p> <p>Copyright (c) 1992-2011 by Mike Gleason.<br/> All rights reserved.</p> <p>Connecting to ftp.hpdd.intel.com...<br/> Welcome to the FTP service on ftp-1.onyx.hpdd.intel.com. Please be nice.<br/> Logging in...<br/> Login successful.<br/> Logged in to ftp.hpdd.intel.com.<br/> ncftp / &gt; cd uploads<br/> Directory successfully changed.<br/> ncftp /uploads &gt; mkdir <a href="https://jira.whamcloud.com/browse/LU-9852" title=" zfs pool creation fails in version 0.7.0 " class="issue-link" data-issue-key="LU-9852"><del>LU-9852</del></a><br/> ncftp /uploads &gt; cd <a href="https://jira.whamcloud.com/browse/LU-9852" title=" zfs pool creation fails in version 0.7.0 " class="issue-link" data-issue-key="LU-9852"><del>LU-9852</del></a><br/> Directory successfully changed.<br/> ncftp /uploads/<a href="https://jira.whamcloud.com/browse/LU-9852" title=" zfs pool creation fails in version 0.7.0 " class="issue-link" data-issue-key="LU-9852"><del>LU-9852</del></a> &gt; ls<br/> ncftp /uploads/<a href="https://jira.whamcloud.com/browse/LU-9852" title=" zfs pool creation fails in version 0.7.0 " class="issue-link" data-issue-key="LU-9852"><del>LU-9852</del></a> &gt; put sosreport-AbeAsraoui-20170811194052.tar.xz<br/> sosreport-AbeAsraoui-20170811194052.tar.xz: 102.83 MB 2.43 MB/s<br/> Could not preserve times for sosreport-AbeAsraoui-20170811194052.tar.xz: UTIME failed.<br/> ncftp /uploads/<a href="https://jira.whamcloud.com/browse/LU-9852" title=" zfs pool creation fails in version 0.7.0 " class="issue-link" data-issue-key="LU-9852"><del>LU-9852</del></a> &gt; ls -l sos*<br/> <del>rw-r</del><del>r</del>- 1 99 50 107827736 Aug 22 2017 sosreport-AbeAsraoui-20170811194052.tar.xz</p> <p>thanks,<br/> Abe</p> <p>Looking at the log, it looks like you had ZFS 0.6.5.9 installed then upgraded to 0.7.0, but the old modules weren't unloaded prior to upgrade, so I think that is where the issue lay, and why a reboot fixed it.</p> <p>make sense, pls go ahead close this ticket..</p> <p>thank you Nathaniel for the support,<br/> Abe</p> Development Rank 1|hzzi7j: Rank (Obsolete) 9223372036854775807 Severity