https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9104 Lustre <p>If we have unknown config parameter in llog then it fails the target mounting.</p> <p>Steps to re-create it are as :</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">$&gt;sh llmount.sh $&gt;export NOFORMAT=yes $&gt;<span class="code-keyword">for</span> OST in $(lctl get_param mgs.MGS.live.lustre | grep OST); <span class="code-keyword">do</span> echo <span class="code-quote">"max_rpcs_in_flight=50 on $OST"</span> ; lctl conf_param $OST.ost.max_rpcs_in_flight=50 ; done &lt;--- Note here ost.max.. is mention instead of osc.max.. $&gt;sh llmountcleanup.sh $&gt;sh llmount.sh &lt;------- <span class="code-keyword">this</span> hangs, as ost is not avilable </pre> </div></div> LU-9104

Unknown config param in llog fails mounting target

Bug Minor Resolved Fixed James A Simmons Rahul Deshmukh Fri, 10 Feb 2017 06:33:49 +0000 Thu, 29 Nov 2018 14:50:45 +0000 Thu, 29 Nov 2018 14:50:45 +0000 Lustre 2.10.1 Lustre 2.11.0 0 7 <p>Rahul Deshmukh (rahul.deshmukh@seagate.com) uploaded a new patch: <a href="https://review.whamcloud.com/25368" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/25368</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9104" title="Unknown config param in llog fails mounting target" class="issue-link" data-issue-key="LU-9104"><del>LU-9104</del></a> tests: check for unknown config param while mounting<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5864f7b957c946dc4726cce180c84e7c2b323eee</p> <p>Pushed the patch and re-producer, please review.</p> <p>James</p> <p>Can you please organize reviews of this proposed test?</p> <p>Thanks</p> <p>Peter</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/25368/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/25368/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9104" title="Unknown config param in llog fails mounting target" class="issue-link" data-issue-key="LU-9104"><del>LU-9104</del></a> obd: Ignore unknown config param while mounting<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 1385dbf8b9caedca7bb32f35db1529e4d5c52d4f</p> <p>Landed for 2.11</p> <p>Minh Diep (minh.diep@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/28232" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28232</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9104" title="Unknown config param in llog fails mounting target" class="issue-link" data-issue-key="LU-9104"><del>LU-9104</del></a> obd: Ignore unknown config param while mounting<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: 902a5fba10096e3196cfed25a2bfc7db0bfe692a</p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/28232/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28232/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9104" title="Unknown config param in llog fails mounting target" class="issue-link" data-issue-key="LU-9104"><del>LU-9104</del></a> obd: Ignore unknown config param while mounting<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: 542815f43c2fbbc29e528fc1de496e54851ca720</p> <p>I am using lustre-2.10.5, when i mount mgs&amp;mdt, it report unkown config parameter, how i remove wrong parameter.</p> <p> </p> <p>```</p> <p><span class="error">&#91;Thu Nov 29 15:00:14 2018&#93;</span> LDISKFS-fs (sdb): mounted filesystem with ordered data mode. Opts: errors=remount-ro<br/> <span class="error">&#91;Thu Nov 29 15:00:17 2018&#93;</span> LDISKFS-fs (sdb): mounted filesystem with ordered data mode. Opts: errors=remount-ro<br/> <span class="error">&#91;Thu Nov 29 15:00:18 2018&#93;</span> LDISKFS-fs (sdb): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> Lustre: MGS: Connection restored to 0b413a06-995f-b510-6aba-9810cdf71a17 (at 0@lo)<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> Lustre: Found index 0 for public4-MDT0000, updating log<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> LustreError: 11-0: public4-OST0000-osc-MDT0000: operation ost_connect to node 10.10.1.11@o2ib failed: rc = -114<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> LustreError: Skipped 4 previous similar messages<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> LustreError: 19825:0:(obd_config.c:1361:class_process_proc_param()) public4-MDT0000: unknown config parameter 'mdt.job_cleanaup_interval=60'<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> LustreError: 19825:0:(obd_config.c:1682:class_config_llog_handler()) MGC10.10.1.14@o2ib: cfg command failed: rc = -38<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> Lustre: cmd=cf00f 0:public4-MDT0000 1:mdt.job_cleanaup_interval=60</p> <p><span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> LustreError: 15c-8: MGC10.10.1.14@o2ib: The configuration from log 'public4-MDT0000' failed (-38). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> LustreError: 19761:0:(obd_mount_server.c:1386:server_start_targets()) failed to start server public4-MDT0000: -38<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> LustreError: 19761:0:(obd_mount_server.c:1879:server_fill_super()) Unable to start targets: -38<br/> <span class="error">&#91;Thu Nov 29 15:00:19 2018&#93;</span> Lustre: Failing over public4-MDT0000<br/> <span class="error">&#91;Thu Nov 29 15:00:25 2018&#93;</span> Lustre: 19761:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: <span class="error">&#91;sent 1543474819/real 1543474819&#93;</span> req@ffff99bab7d6a100 x1618446900006864/t0(0) o251-&gt;MGC10.10.1.14@o2ib@0@lo:26/25 lens 224/224 e 0 to 1 dl 1543474825 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1<br/> <span class="error">&#91;Thu Nov 29 15:00:26 2018&#93;</span> Lustre: server umount public4-MDT0000 complete<br/> <span class="error">&#91;Thu Nov 29 15:00:26 2018&#93;</span> LustreError: 19761:0:(obd_mount.c:1582:lustre_fill_super()) Unable to mount (-38)</p> <p>```</p> <p>I am trying to use `mount -t lustre -o nosvc xxx` which only start mgs, then execute `</p> <p>lctl conf_param -d public4-MDT0000.mdt.job_cleanaup_interval` to remove wrong config.</p> <p>The patch of <a href="https://review.whamcloud.com/25368/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/25368/</a> might still has a bug in osd_process_config()</p> <p>When a llog config of type MDT can not be understood by class_process_proc_param(PARAM_MDT) or any other prefix (e.g. PARAM_HSM), it is passed to osd_process_config() and class_process_proc_param() is called for it for twice.</p> <p>The problem is why the -ENOSYS failure of class_process_proc_param(PARAM_OST) is not ignored in osd_process_config()?</p> <p>I am not sure about the llog processing codes, but I guess the process is: class_process_proc_param(PARAM_MDT) -&gt; class_process_proc_param(PARAM_HSM) -&gt; class_process_proc_param(PARAM_HSM) -&gt; class_process_proc_param(PARAM_OSD) -&gt; class_process_proc_param(PARAM_OST). So if osd_process_config() doesn't understand the llog record, it should ignore the -ENOSYS. Right?</p> <p>I think we should fix this problem by ignoring ENOSYS in osd_process_config() so reopening the ticket.</p> <p>IMHO, given that this bug has existed in multiple shipped releases, it would be better to track this issue in a new ticket so that we can more easily track getting it into future releases.</p> <p>I can fix this under <a href="https://jira.whamcloud.com/browse/LU-8066" title="Move lustre procfs handling to sysfs and debugfs." class="issue-link" data-issue-key="LU-8066">LU-8066</a> since we will be moving to sysfs which handles these error codes differently.</p> <p>great - thanks James</p> Duplicate Related LU-8066 Development Rank 1|hzz3fz: Rank (Obsolete) 9223372036854775807 Severity