https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14108 Lustre <p>The <tt>--network=</tt> option to mkfs.lustre allows restricting a target (OST/MDT) to a given LNet network. This makes it register to the MGS with the specified network only. However, dynamic discovery is unaware of this restriction and this can create problems.</p> <p>If this scenario is recognised, it is possible to deal with it in at least two ways:</p> <p>1)  Prevent the mount with an error.</p> <p>2) Disable dynamic discovery with a warning, prevent from enabling discovery in the future.</p> LU-14108

Mounting targets created with mkfs "network" option should disable discovery

Bug Minor Resolved Fixed Cyril Bordage Serguei Smirnov lnet Mon, 2 Nov 2020 19:35:19 +0000 Tue, 30 May 2023 08:33:10 +0000 Tue, 30 May 2023 08:33:10 +0000 0 3 <p>To reproduce, use --network option when formatting targets with mkfs.lustre. The following is an example from the customer setup:</p> <p>"</p> <p><em>The error is occurring between MDS (md1) and OSS (all OSS showing similar errors). For example, in the case of rcf2-OST000d, it is being mounted on os2, but md1 is trying to connect it to os1 repeatedly. Usually rcf2-OST000d is mounted on os2 (primary OSS for this OST) and error should not happen.</em></p> <p><em>tunefs.lustre shows network=o2ib1, and primary is os2 (172.20.2.26@o2ib1) and secondary is os1 (172.20.2.25@o2ib1)</em></p> <p>"</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> [root@os2 ~]# tunefs.lustre --dryrun /dev/mapper/OST0d checking <span class="code-keyword">for</span> existing Lustre data: found Read previous values: Target: rcf2-OST000d Index: 13 Lustre FS: rcf2 Mount type: ldiskfs Flags: 0x2 (OST ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=172.20.2.11@o2ib:172.20.2.12@o2ib failover.node=172.20.2.25@o2ib1 network=o2ib1 Permanent disk data: Target: rcf2-OST000d Index: 13 Lustre FS: rcf2 Mount type: ldiskfs Flags: 0x2 (OST ) Persistent mount opts: errors=remount-ro Parameters: mgsnode=172.20.2.11@o2ib:172.20.2.12@o2ib failover.node=172.20.2.25@o2ib1 network=o2ib1 exiting before disk write.</pre> </div></div> <p>It should be possible to reproduce with a simpler setup. </p> <p>The symptom is the error message similar to the following:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> Oct 22 11:58:27 os1-s kernel: Lustre: rcf2-OST0001: Received <span class="code-keyword">new</span> MDS connection from 172.20.2.11@o2ib, removing former export from same NID Oct 22 11:58:27 os1-s kernel: Lustre: Skipped 47 previous similar messages ... Oct 22 12:02:13 os1-s kernel: LustreError: 137-5: rcf2-OST000d_UUID: not available <span class="code-keyword">for</span> connect from 172.20.2.11@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. Oct 22 12:02:13 os1-s kernel: LustreError: Skipped 47 previous similar messages</pre> </div></div> <p>To see how the targets are registered to the MGS, run command similar to the following on the MGS:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> lctl --device MGS llog_print rcf2-MDT0000</pre> </div></div> <p>This should show that only NIDs on the network specified as the option with mkfs were used.</p> <p>"Cyril Bordage &lt;cbordage@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/46632" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/46632</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14108" title="Mounting targets created with mkfs &quot;network&quot; option should disable discovery" class="issue-link" data-issue-key="LU-14108"><del>LU-14108</del></a> mount: prevent if --network and discovery<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: f5f0dba9c253a25ba9b6d06a0b4e361aa48690d5</p> <p>"Cyril Bordage &lt;cbordage@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/47137" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47137</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14108" title="Mounting targets created with mkfs &quot;network&quot; option should disable discovery" class="issue-link" data-issue-key="LU-14108"><del>LU-14108</del></a> tests: mount with discovery and network<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 038db7e1872e6ef743921744fb09f0f0c201124a</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/46632/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/46632/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14108" title="Mounting targets created with mkfs &quot;network&quot; option should disable discovery" class="issue-link" data-issue-key="LU-14108"><del>LU-14108</del></a> mount: prevent if --network and discovery<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: e82836f56ee7a9337a86ad0a32f19751024c7ec6</p> Related LU-11057 Development Rank 1|i01e33: Rank (Obsolete) 9223372036854775807 Severity