https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14061 Lustre <p><a href="https://jira.whamcloud.com/browse/LU-12514" title="separate out the lustre mount code for server and clients" class="issue-link" data-issue-key="LU-12514">LU-12514</a> (utils: add "lustre_tgt" filesystem type) breaks scripts that used 'mount -t lustre' and expected to see targets.</p> <p>In 2.12.*:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># mount -t lustre /dev/mapper/mds1_flakey on /mnt/lustre-mds1 type lustre (ro,svname=lustre-MDT0000,mgs,osd=osd-ldiskfs,user_xattr,errors=remount-ro) /dev/mapper/ost1_flakey on /mnt/lustre-ost1 type lustre (ro,svname=lustre-OST0000,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) /dev/mapper/ost2_flakey on /mnt/lustre-ost2 type lustre (ro,svname=lustre-OST0001,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) /dev/mapper/ost3_flakey on /mnt/lustre-ost3 type lustre (ro,svname=lustre-OST0002,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) /dev/mapper/ost4_flakey on /mnt/lustre-ost4 type lustre (ro,svname=lustre-OST0003,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) 192.168.122.75@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr,lazystatfs) # mount -t lustre_tgt # </pre> </div></div> <p>In 2.13 and on:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># mount -t lustre 192.168.122.75@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr,lazystatfs,noencrypt) # mount -t lustre_tgt /dev/mapper/ost1_flakey on /mnt/lustre-ost1 type lustre_tgt (ro,svname=lustre-OST0000,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) /dev/mapper/ost2_flakey on /mnt/lustre-ost2 type lustre_tgt (ro,svname=lustre-OST0001,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) /dev/mapper/ost3_flakey on /mnt/lustre-ost3 type lustre_tgt (ro,svname=lustre-OST0002,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) /dev/mapper/ost4_flakey on /mnt/lustre-ost4 type lustre_tgt (ro,svname=lustre-OST0003,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) /dev/mapper/mds1_flakey on /mnt/lustre-mds1 type lustre_tgt (ro,svname=lustre-MDT0000,mgs,osd=osd-ldiskfs,user_xattr,errors=remount-ro) </pre> </div></div> LU-14061

LU-12514 (lustre_tgt mount type) breaks scripts that used 'mount -t lustre' and expected targets

Bug Major Resolved Fixed Andreas Dilger John Hammond Wed, 21 Oct 2020 17:24:38 +0000 Thu, 19 Nov 2020 14:39:58 +0000 Thu, 19 Nov 2020 14:39:58 +0000 Lustre 2.13.0 Lustre 2.14.0 0 5 <p>The solution is to update to 'mount -t lustre,lustre_tgt'. NFS did the same thing for NFSv4 support.  The idea of a 'lustre' file system type for both servers and clients works well since the normal setup is separate nodes for clients and servers. If mounting a light weight client will be the norm in the future on server nodes I can see a 'lustre' for all leading to problems.</p> <p>This is not a solution for existing scripts.</p> <p>John, how were the server filesystems mounted originally? Did they get mounted with "<tt>mount -t lustre</tt>" or "<tt>mount -t lustre_tgt</tt>" initially? Since 2.13, "<tt>mount -t lustre</tt>" will internally call "<tt>mount.lustre</tt>" and "<tt>mount -t lustre_tgt</tt>" will internally call "<tt>mount.lustre_tgt</tt>", but both are the same binary and will try "<tt>lustre_tgt</tt>" first for server mountpoints (block devices) and then fall back to using "<tt>lustre</tt>" if that doesn't work. I tested a trivial change to "<tt>mount_lustre.c</tt>" that only uses "<tt>lustre</tt>", and "mount -t lustre" now works fine. It would probably be trivial to distinguish via "<tt>argv<span class="error">&#91;0&#93;</span></tt>" which binary is being called and use that preferentially so that the mounted type matches what is requested.</p> <p>Since commit <tt>v2_12_58-71-g510aea4a37</tt>, the use of <tt>lustre_tgt</tt> is optional so it is still possible to mount a server as type <tt>lustre</tt> to give some time to migrate system configurations. We definitely want to keep this compatibility until <em>at least the next LTS</em>, so that there is at least one commonly-used release where it is possible to use either type so upgrade/downgrade is possible without a flag day for changing the configuration at the same time. Possibly for 2.14 or 2.15 it seems reasonable to add a message to <tt>mount.lustre</tt> and/or the kernel to warn if <tt>lustre</tt> is used for a server mountpoint, but that wouldn't cause any errors for users. </p> <p>John also pointed out that "<tt>stat -f</tt>" should handle this as well. This will currently report "<tt>UNKNOWN (0xbd00bd1)</tt>" as the filesystem magic (<tt>LUSTRE_SUPER_MAGIC</tt>), regardless of what type it was mounted as. This is also true on my 2.10.8 server, so it is not a regression introduced by <a href="https://jira.whamcloud.com/browse/LU-12514" title="separate out the lustre mount code for server and clients" class="issue-link" data-issue-key="LU-12514">LU-12514</a>. It makes sense to submit an upstream patch to fix this to return "<tt>lustre_tgt</tt>" so that it is correct by the time it makes its way into the vendor distros. If anyone is using bleeding-edge <tt>coreutils</tt> it is likely that they are also using a newer Lustre release, or this is on the client where it doesn't matter since it returns magic "<tt>LL_SUPER_MAGIC = 0x0bd00bd0</tt>" which is mapped to "<tt>lustre</tt>" in userspace.</p> <p>&gt; how were the server filesystems mounted originally? Did they get mounted with "mount -t lustre" or "mount -t lustre_tgt" initially?</p> <p>Using <tt>mount -t lustre</tt>.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>k:~# mount -t lustre 192.168.122.75@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr,lazystatfs,noencrypt) k:~# mount -t lustre_tgt ... /dev/mapper/ost1_flakey on /mnt/lustre-ost1 type lustre_tgt (ro,svname=lustre-OST0000,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) k:~# umount /mnt/lustre-ost1/ k:~# mount /dev/mapper/ost1_flakey /mnt/lustre-ost1 -t lustre k:~# mount -t lustre 192.168.122.75@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr,lazystatfs,noencrypt) k:~# mount -t lustre_tgt ... /dev/mapper/ost1_flakey on /mnt/lustre-ost1 type lustre_tgt (ro,svname=lustre-OST0000,mgsnode=192.168.122.75@tcp,osd=osd-ldiskfs,errors=remount-ro) </pre> </div></div> <p>So this appears to be a bug in <tt>mount_lustre.c</tt> that it is <b>always</b> using "<tt>lustre_tgt</tt>" for server targets instead of what was requested:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">if</span> (!mop.mo_fake) { <span class="code-object">char</span> *fstype = client ? <span class="code-quote">"lustre"</span> : <span class="code-quote">"lustre_tgt"</span>; /* * flags and target get to lustre_get_sb(), but not * lustre_fill_super(). Lustre ignores the flags, but mount * does not. */ <span class="code-keyword">for</span> (i = 0, rc = -EAGAIN; i &lt;= mop.mo_retry &amp;&amp; rc != 0; i++) { rc = mount(mop.mo_source, mop.mo_target, fstype, flags, (void *)options); <span class="code-keyword">if</span> (rc != 0) { /* Older Lustre without <span class="code-quote">'lustre_tgt'</span>. * Try <span class="code-quote">'lustre'</span> instead */ <span class="code-keyword">if</span> (rc == -ENODEV) { fstype = <span class="code-quote">"lustre"</span>; i--; <span class="code-keyword">continue</span>; } </pre> </div></div> <p>This should check <tt>basename(argv<span class="error">&#91;0&#93;</span>)</tt> to see if it is called as "<tt>mount.lustre</tt>" and preferably use "<tt>lustre</tt>" in that case, and "<tt>mount.lustre_tgt</tt>" should preferably use "<tt>lustre_tgt</tt>" then. For server mountpoints it could also try "<tt>lustre_tgt</tt>" if the first mount fails with <tt>-ENODEV</tt>.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">if</span> (!mop.mo_fake) { <span class="code-object">char</span> *fstype = <span class="code-quote">"lustre"</span>; <span class="code-keyword">if</span> (strstr(argv[0], <span class="code-quote">"mount.lustre_tgt"</span>)) fstype = <span class="code-quote">"lustre_tgt"</span>; /* * flags and target get to lustre_get_sb(), but not * lustre_fill_super(). Lustre ignores the flags, but mount * does not. */ <span class="code-keyword">for</span> (i = 0, rc = -EAGAIN; i &lt;= mop.mo_retry &amp;&amp; rc != 0; i++) { rc = mount(mop.mo_source, mop.mo_target, fstype, flags, (void *)options); <span class="code-keyword">if</span> (rc != 0) { /* Older Lustre without <span class="code-quote">'lustre_tgt'</span>. * Try <span class="code-quote">'lustre'</span> instead */ <span class="code-keyword">if</span> (rc == -ENODEV &amp;&amp; strcmp(fstype, <span class="code-quote">"lustre_tgt"</span>)) { fstype = <span class="code-quote">"lustre"</span>; i--; <span class="code-keyword">continue</span>; } </pre> </div></div> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/40474" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40474</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14061" title="LU-12514 (lustre_tgt mount type) breaks scripts that used &#39;mount -t lustre&#39; and expected targets" class="issue-link" data-issue-key="LU-14061"><del>LU-14061</del></a> utils: prefer mounting with specified fstype<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a79f8111eb6ad27d42a4a04cc9f20e74bdbcbc8c</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/40474/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40474/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14061" title="LU-12514 (lustre_tgt mount type) breaks scripts that used &#39;mount -t lustre&#39; and expected targets" class="issue-link" data-issue-key="LU-14061"><del>LU-14061</del></a> utils: prefer mounting with specified fstype<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: ffeb900743772d86ceb41b4a323006fd478fd674</p> <p>Landed for 2.14</p> Related LU-12514 Development Rank 1|i01cxb: Rank (Obsolete) 9223372036854775807 Severity