https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-16164 Lustre <p>From <a href="https://review.whamcloud.com/#/c/fs/lustre-release/+/48417/8/lustre/tests/sanity.sh@1154" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/fs/lustre-release/+/48417/8/lustre/tests/sanity.sh@1154</a></p> <p>It would be good to run "lctl get_param -R '*'" on both the client and server to see which parameters are in YAML format, and add checks for them so that they don't break in the future.</p> LU-16164

Check params and make them valid YAML if they are designed to be

Improvement Minor Open Unresolved Feng Lei Feng Lei Fri, 16 Sep 2022 01:00:04 +0000 Thu, 2 Nov 2023 06:12:10 +0000 0 2 <p>Check all the params with a script:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> #!/bin/bash    &gt;read_fail_list &gt;invalid_yaml_list <span class="code-keyword">for</span> i in `lctl get_param -R -N <span class="code-quote">"*"</span>`; <span class="code-keyword">do</span>         echo ===$i===         lctl get_param $i &gt; /dev/<span class="code-keyword">null</span> || echo $i &gt;&gt; read_fail_list         lctl get_param -n $i | yq         <span class="code-keyword">if</span> [[ $? != 0 ]]; then                 echo $i &gt;&gt; invalid_yaml_list         fi done </pre> </div></div> <p>Get 2 files lists, one for files cannot be read, the other for invalid YAML. In the attachment.</p> <p>Currently there are 75 files cannot be read, and 51 invalid YAML files.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> # wc read_fail_list    75   75 2733 read_fail_list # wc invalid_yaml_list    51   51 1844 invalid_yaml_list </pre> </div></div> <p>Even some files pass the check of yq, they are parsed as a single string, not meaningful YAML. For example:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> # lctl get_param -n mgs.*.*.stats snapshot_time             1663721269.216694664 secs.nsecs req_waittime              15358 samples [usec] 12 67506 3474626 11719248936 req_qdepth                15358 samples [reqs] 0 0 0 0 ... </pre> </div></div> <p>It's actually parsed as a single string, not a meaningful YAML:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> # lctl get_param -n mgs.*.*.stats | yq <span class="code-quote">"snapshot_time             1663721319.904973718 secs.nsecs req_waittime              15368 samples [usec] 12 67506 3477317 11720202797 req_qdepth                15368 samples [reqs] 0 0 0 0 req_active                15368 samples [reqs] 1 2 15369 15371 req_timeout               15368 samples [sec] 1 10 15377 15467 reqbuf_avail              45997 samples [bufs] 62 64 2913042 184496438 ldlm_plain_enqueue        50 samples [reqs] 1 1 50 50 mgs_connect               1 samples [usec] 156 156 156 24336 mgs_target_reg            4 samples [usec] 573 8409 17020 105176030 mgs_config_read           9 samples [usec] 125 959 4866 3121456 obd_ping                  15151 samples [usec] 2 5710 866463 192267777 llog_origin_handle_open   39 samples [usec] 99 1632 13234 7631248 llog_origin_handle_next_block 76 samples [usec] 92 1099 25799 12133241 llog_origin_handle_read_header 38 samples [usec] 71 1073 11972 5535674 snapshot_time             1663721319.905164765 secs.nsecs"</span> </pre> </div></div> <p>The "<tt>&#42;.&#42;.stats</tt>" files are mostly <b>not</b> YAML formatted.</p> <p>The <tt>osc/mdc.&#42;.import</tt> file should be YAML, as with the <tt>obdfilter/mdt.&#42;.exports.&#42;.export</tt> files.</p> <p>It looks like on the server there are <tt>ldlm.services.ldlm_canceld.nrs_policies</tt> and <tt>ldlm.services.ldlm_cbd.nrs_policies</tt>, along with <tt>ost.OSS.&#42;.nrs_policies</tt>, <tt>mds.MDS.&#42;.nrs_policies</tt> and <tt>mgs.MGS.mgs.nrs_policies</tt>, <tt>mdd.&#42;.lfsck_layout</tt>, <tt>mdt.&#42;.exports.&#42;.reply_data</tt>, <tt>mdt.&#42;.recovery_status</tt> files.</p> <p>There is also the quota files <tt>osd-ldiskfs.&#42;.quota_slave&#42;.acct_group</tt>, <tt>osd-ldiskfs.&#42;.quota_slave&#42;.acct_group,acct_project,acct_user</tt>, <tt>osd-ldiskfs.&#42;.quota_slave&#42;.info</tt>, <tt>osd-ldiskfs.&#42;.quota_slave&#42;.limit_group,limit_project,limit_user</tt>, <tt>osd-ldiskfs.&#42;.oi_scrub</tt>, <tt>qmt.&#42;.dt-0x0.glb-grp,glb-prj,glb-usr</tt> .</p> <p>On the client, <tt>llite.&#42;.max_cached_mb</tt>, <tt>llite.&#42;.statahead_stats</tt>, <tt>llite,osc,mdc.&#42;.unstable_stats</tt>, <tt>osc.&#42;.osc_cached_mb</tt>, <tt>mdc.&#42;.mdc_cached_mb</tt>, <tt>mdc.&#42;.state</tt>.</p> <p>The <tt>lwp.&#42;.srpc_info</tt> looks like it could/should be in YAML format, but needs a few small improvements (colon after "<tt>gc internal</tt>" and "<tt>gc next</tt>").</p> Related LU-16110 Development Rank 1|i030cn: Rank (Obsolete) 9223372036854775807