https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-15862 Lustre <p>We have a critical Mellanox firmware issue causing HCA soft lockups. As a possible work around we are considering using  tcp NIDs, until we have a fix from Mellanox.</p> <p>We want to configure servers with tcp and o2ib nids, doing so will avoid future down time switching NIDS on the server.</p> <p>Is there a way to select/prefer NIDs on the client per filesystem?</p> <p>Here is an example config:</p> <p>client_a:  <span class="error">&#91;o2ib(ib0) tcp(ib0)&#93;</span> (mount fs1 using tcp0) (mount fs2 using o2ib) (mount fs3 mount o2ib)</p> <p>client_b: <span class="error">&#91;o2ib414(ib0)&#93;</span>  (mount fs1 using tcp0) (mount fs2 using o2ib) (mount fs3 mount o2ib)</p> <p>lrouter:  o2ib41(ib1) :: o2ib(ib0) tcp(ib0)</p> <p> </p> <p>fs1-srv1:   o2ib(ib0) tcp(ib0)</p> <p>fs2-srv1:  o2ib(ib0) tcp(ib0)</p> <p>fs3-srv1: o2ib(ib0)</p> LU-15862

How to set prefer server NID

Question/Request Critical Open Unresolved Serguei Smirnov Mahmoud Hanafi Tue, 17 May 2022 00:09:50 +0000 Fri, 10 Jun 2022 22:20:59 +0000 Lustre 2.12.8 0 8 <p>Serguei, can you please comment and/or sub-assign. </p> <p>Hi <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=mhanafi" class="user-hover" rel="mhanafi">mhanafi</a>,</p> <p>We're actively looking into this, would you mind if I ask, which mellanox firmware critical issue are you dealing with?</p> <p>-cf</p> <p> </p> <p>Hi Mahmoud,</p> <p>We implemented the UDSP feature. This allows adding rules to do what you're looking for. This feature was added in Lustre 2.15</p> <p>Here is how it would work in your case</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> lnetctl udsp add o2ib0 --priority 0 lnetctl udsp add tcp0 --priority 1</pre> </div></div> <p>0 is the highest priority. This will prefer o2ib0 always unless o2ib0 becomes unreachable in which case you'd start using tcp0.</p> <p>If you add both of these rules on all nodes, then o2ib0 will always be preferred. Through the health feature, LNet will detect if this network is not reachable or peers on this network are not reachable and start using tcp0.</p> <p>This feature is built on the Multi-Rail discovery features. So discovery should be on to allow the nodes to associate both interfaces to the same peer.</p> Development Rank 1|i02q5b: Rank (Obsolete) 9223372036854775807