https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-15538 Lustre <p>LND tunables are not initialized correctly when any net or lnd tunable parameter is specified via either the CLI or via the yaml config.</p> <p>I noticed this with map_on_demand, but the bug potentially impacts any LND tunable where '0' is an accepted value:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>/home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net show -v 5 | egrep -e o2ib -e map - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 1 /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 --peer-credits=32 /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net show -v 5 | egrep -e o2ib -e map - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 0 </pre> </div></div> <p>Issue is that when any NET or LND tunable is specified via CLI or yaml, then the whole tunables struct gets memset to 0, or in the case of yaml config, 0 gets assigned to any tunable that isn't specified in the yaml. LND then thinks that '0' was specified by user and will use that value (if it is valid).</p> LU-15538

DLC doesn't initialize default LND tunables correctly

Bug Minor Resolved Fixed Chris Horn Chris Horn Tue, 8 Feb 2022 22:08:04 +0000 Fri, 23 Sep 2022 16:41:54 +0000 Sat, 11 Jun 2022 15:41:23 +0000 Lustre 2.16.0 0 3 <p>"Chris Horn &lt;chris.horn@hpe.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/46492" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/46492</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15538" title="DLC doesn&#39;t initialize default LND tunables correctly" class="issue-link" data-issue-key="LU-15538"><del>LU-15538</del></a> lnet: DLC sets map_on_demand incorrectly<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 6d5b0308cae0db67daa7df626a60fd0825d252cc</p> <p>Test notes for <a href="https://jira.whamcloud.com/browse/LU-15538" title="DLC doesn&#39;t initialize default LND tunables correctly" class="issue-link" data-issue-key="LU-15538"><del>LU-15538</del></a>:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[hornc@s-lmo-gaz38b lustre-wc-rel]$ git reset --hard origin/master HEAD is now at 450d10c362 New RC 2.15.0-RC2 [hornc@s-lmo-gaz38b lustre-wc-rel]$ make -j 32 ... </pre> </div></div> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@s-lmo-gaz38b hornc]# bash start.sh /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 1 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh map_on_demand=0 /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 0 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh map_on_demand=1 /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 1 [root@s-lmo-gaz38b hornc]# </pre> </div></div> <p>We can see map_on_demand has expected value (note, the default is 1) in each case.</p> <p>Repeat test, but specify '--peer-credits=32':</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@s-lmo-gaz38b hornc]# bash start.sh2 debug=+net /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 --peer-credits=32 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 0 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh2 map_on_demand=0 debug=+net /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 --peer-credits=32 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 0 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh2 map_on_demand=1 debug=+net /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 --peer-credits=32 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 0 [root@s-lmo-gaz38b hornc]# </pre> </div></div> <p>We can see the wrong value is used in 2/3 cases.</p> <p>Apply fix:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[hornc@s-lmo-gaz38b lustre-wc-rel]$ git fetch https://review.whamcloud.com/fs/lustre-release refs/changes/92/46492/1 &amp;&amp; git cherry-pick FETCH_HEAD remote: Counting objects: 2010, done remote: Finding sources: 100% (10/10) remote: Total 10 (delta 9), reused 10 (delta 9) Unpacking objects: 100% (10/10), 1.30 KiB | 665.00 KiB/s, done. From https://review.whamcloud.com/fs/lustre-release * branch refs/changes/92/46492/1 -&gt; FETCH_HEAD [tmp4 c2c7abe9b6] LU-15538 lnet: DLC sets map_on_demand incorrectly Date: Sat Feb 5 23:15:30 2022 +0000 3 files changed, 7 insertions(+), 1 deletion(-) [hornc@s-lmo-gaz38b lustre-wc-rel]$ make -j 32 ... </pre> </div></div> <p>Repeat test cases:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@s-lmo-gaz38b hornc]# bash start.sh /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 1 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh map_on_demand=0 /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 0 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh map_on_demand=1 /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 1 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh2 debug=+net /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 --peer-credits=32 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 1 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh2 map_on_demand=0 debug=+net /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 --peer-credits=32 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 0 [root@s-lmo-gaz38b hornc]# bash clean.sh [root@s-lmo-gaz38b hornc]# bash start.sh2 map_on_demand=1 debug=+net /home/hornc/lustre-wc-rel/lnet/utils/lnetctl lnet configure /home/hornc/lustre-wc-rel/lnet/utils/lnetctl net add --net o2ib --if enp137s0f0 --peer-credits=32 - net type: o2ib - nid: 172.18.4.4@o2ib map_on_demand: 1 [root@s-lmo-gaz38b hornc]# </pre> </div></div> <p>We can see map_on_demand gets correct value in every case.</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/46492/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/46492/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15538" title="DLC doesn&#39;t initialize default LND tunables correctly" class="issue-link" data-issue-key="LU-15538"><del>LU-15538</del></a> lnet: DLC sets map_on_demand incorrectly<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 896f4a082b93453f5e7168f685faff4fba594ff3</p> <p>Landed for 2.16</p> Development Rank 1|i02hvj: Rank (Obsolete) 9223372036854775807 Severity