https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10216 Lustre <p>Hi All,</p> <p>There are 4 X OSS servers and 2 X MGS/MDS in our Lustre filesystem. </p> <p>We are trying to enable NRS_TBF by using following instructions in our MGS server.</p> <p><span class="image-wrap" style=""><img src="https://jira.whamcloud.com/secure/attachment/28613/28613_Selection_019.png" style="border: 0px solid black" /></span></p> <p>After this, we want to verify if these rules are added successfully by using following instructions in one OSS server. The result is shown below.</p> <p><span class="image-wrap" style=""><img src="https://jira.whamcloud.com/secure/attachment/28614/28614_Selection_020.png" style="border: 0px solid black" /></span>  </p> <p>Is it correct?  If no, please kindly provide us with instructions to do this.</p> <p>Also, how can we verify if these rules are "Change" and "Stop(Delete)" successfully when we use "lctl set_param ost.OSS.ost_io.nrs_tbf_rule="change ossb1 500" and "lctl set_param ost.OSS.ost_io.nrs_tbf_rule="stop ossb1", respectively.  </p> <p> </p> <p>Thanks!</p> Lustre 2.10.0/ Centos 7.3 LU-10216

One question about enabling NRS_TBF in Lustre

Question/Request Critical Resolved Fixed Emoly Liu sebg-crd-pm Thu, 9 Nov 2017 07:10:50 +0000 Fri, 20 Jul 2018 02:35:10 +0000 Fri, 20 Jul 2018 02:35:10 +0000 0 4 <p>Emoly</p> <p>Can you please advise?</p> <p>Thanks</p> <p>Peter</p> <p>Hi Emoly, </p> <p>Please give us some suggestions for this issue.</p> <p>We really need your help.</p> <p>Thanks!</p> <p> </p> <p>Hi <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=sebg-crd-pm" class="user-hover" rel="sebg-crd-pm">sebg-crd-pm</a>,</p> <p>The TBF policy format has been changed since lustre 2.8.55, as follows</p> <ul class="alternate" type="square"> <li>For NRS "tbf nid" policy:<br/> lctl set_param ost.OSS.ost_io.nrs_tbf_rule="start $RULE_NAME nid={$NID} rate=$RATE"</li> <li>For NRS "tbf jobid" policy:<br/> lctl set_param ost.OSS.ost_io.nrs_tbf_rule="start $RULE_NAME jobid={$JOBID} rate=$RATE"</li> <li>To change the policy:<br/> lctl set_param ost.OSS.ost_io.nrs_tbf_rule="change $RULE_NAME rate=$RATE"</li> </ul> <p>So you should change your commands format to make them work. Here is my output for your reference:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@centos7-3 tests]# lctl set_param ost.OSS.ost_io.nrs_policies="tbf nid" ost.OSS.ost_io.nrs_policies=tbf nid [root@centos7-3 tests]# lctl set_param ost.OSS.ost_io.nrs_tbf_rule="start ossb1 nid={172.20.110.209@o2ib} rate=100" ost.OSS.ost_io.nrs_tbf_rule=start ossb1 nid={172.20.110.209@o2ib} rate=100 [root@centos7-3 tests]# lctl set_param ost.OSS.ost_io.nrs_tbf_rule="start robin nid={172.20.110.212@o2ib} rate=200" ost.OSS.ost_io.nrs_tbf_rule=start robin nid={172.20.110.212@o2ib} rate=200 [root@centos7-3 tests]# lctl get_param ost.OSS.ost_io.nrs_tbf_rule ost.OSS.ost_io.nrs_tbf_rule= regular_requests: CPT 0: ossb1 {172.20.110.209@o2ib} 100, ref 0 robin {172.20.110.212@o2ib} 200, ref 0 default {*} 10000, ref 0 high_priority_requests: CPT 0: ossb1 {172.20.110.209@o2ib} 100, ref 0 robin {172.20.110.212@o2ib} 200, ref 0 default {*} 10000, ref 0 [root@centos7-3 tests]# lctl set_param ost.OSS.ost_io.nrs_tbf_rule="change ossb1 rate=500" ost.OSS.ost_io.nrs_tbf_rule=change ossb1 rate=500 [root@centos7-3 tests]# lctl get_param ost.OSS.ost_io.nrs_tbf_rule ost.OSS.ost_io.nrs_tbf_rule= regular_requests: CPT 0: ossb1 {172.20.110.209@o2ib} 500, ref 0 robin {172.20.110.212@o2ib} 200, ref 0 default {*} 10000, ref 0 high_priority_requests: CPT 0: ossb1 {172.20.110.209@o2ib} 500, ref 0 robin {172.20.110.212@o2ib} 200, ref 0 default {*} 10000, ref 0 [root@centos7-3 tests]# lctl set_param ost.OSS.ost_io.nrs_tbf_rule="stop ossb1" ost.OSS.ost_io.nrs_tbf_rule=stop ossb1 [root@centos7-3 tests]# lctl get_param ost.OSS.ost_io.nrs_tbf_rule ost.OSS.ost_io.nrs_tbf_rule= regular_requests: CPT 0: robin {172.20.110.212@o2ib} 200, ref 0 default {*} 10000, ref 0 high_priority_requests: CPT 0: robin {172.20.110.212@o2ib} 200, ref 0 default {*} 10000, ref 0 </pre> </div></div> <p>HTH.<br/> Emoly</p> <p>Please note that "<tt>lctl set_param</tt>" commands are local to the server they are run on, and only in effect while the server is mounted, so these commands need to be run on all of the OSS nodes.</p> <p>It looks like there is an open ticket for the NRS TBF documentation to be updated for the new functionality added by DDN - <a href="https://jira.whamcloud.com/browse/LUDOC-328" title="documentation updates for complex TBF policies" class="issue-link" data-issue-key="LUDOC-328">LUDOC-328</a>.</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=sebg-crd-pm" class="user-hover" rel="sebg-crd-pm">sebg-crd-pm</a>,<br/> Can we close this issue ?<br/> Thanks,<br/> Emoly</p> <p>You can close it. Thanks.</p> <p>Thanks.</p> Related LUDOC-328 Development Rank 1|hzznj3: Rank (Obsolete) 9223372036854775807