https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10906 Lustre <p>Attempting to run</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>mgs$ lctl set_param -P osc.s1lcl*.checksums=0 </pre> </div></div> <p>to disable checksums for their clients does not return any errors, and a</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>mgs$ lctl get_param osc.s1lcl*.checksums </pre> </div></div> <p>shows that it is properly set to 0 at that time. However, after a reboot of a client node, this parameter gets reset to 1. This behavior does not appear for any of the other parameters that they are setting - just the checksums. There is something preventing the <tt>checksums</tt> parameter from being set permanently.</p> LU-10906

checksums parameter not persistent after reboot

Bug Minor Resolved Fixed Emoly Liu Andreas Dilger Wed, 11 Apr 2018 22:20:01 +0000 Mon, 7 Jan 2019 19:19:07 +0000 Sat, 6 Oct 2018 13:29:38 +0000 Lustre 2.10.0 Lustre 2.12.0 0 5 <p>It isn't clear why the <tt>lctl set_param -P</tt> setting is not persistent for this parameter. It might be caused by ordering issues during client mount, since there is a separate "<tt>-o checksum</tt>" and "<tt>-o nochecksum</tt>" mount option available (which is enabled by default) that is also trying to enable the checksum during mount.</p> <p>Mounting the clients with "-o nochecksum" is verified to be working as expected (<tt>lctl get_param osc.*.checksums</tt> returns 0 in this case, and returns 1 without this parameter).</p> <p>Some notes:</p> <ul class="alternate" type="square"> <li>this is likely related to the handling of <tt>LL_SBI_CHECKSUM</tt> during mount clobbering the <tt>osc.*.checksums=0</tt> parameter stored in the configuration. It looks like the OSC <tt>cl_checksum</tt> is already enabled by default in <tt>client_obd_setup()</tt> so there is little benefit to set it again from <tt>client_common_fill_super()</tt> at mount time, so fixing this issue may be as simple as making the following lines conditional on the <tt>checksum</tt> or <tt>nochecksum</tt> options actually being passed, rather than always overriding the OSC settings: <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> checksum = sbi-&gt;ll_flags &amp; LL_SBI_CHECKSUM; err = obd_set_info_async(NULL, sbi-&gt;ll_dt_exp, sizeof(KEY_CHECKSUM), KEY_CHECKSUM, sizeof(checksum), &amp;checksum, NULL); <span class="code-keyword">if</span> (err) { CERROR(<span class="code-quote">"%s: Set checksum failed: rc = %d\n"</span>, sbi-&gt;ll_dt_exp-&gt;exp_obd-&gt;obd_name, err); GOTO(out_root, err); } </pre> </div></div></li> <li>the <tt>llite.&#42;.checksum_pages</tt> tunable is misnamed. This was (in 1.8) a tunable to enable per-page checksums at the llite layer, independent of the BRW RPC data checksums. It would be better to rename this <tt>llite.&#42;.checksums</tt> to match <tt>osc.&#42;.checksums</tt>, and leave <tt>checksum_pages</tt> for its original purpose of enabling per-page checksums (which was not implemented in the CLIO development).</li> </ul> <p>As Andreas said,  after a remount, this parameter is really set to 0 by "process_param2_config()) lctl: invoked upcall /usr/sbin/lctl set_param osc.lustre*.checksums=0", but it is changed later by</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>client_common_fill_super() -&gt;obd_set_info_async() -&gt;lov_set_info_async() -&gt;osc_set_info_async() int osc_set_info_async() { ... if (KEY_IS(KEY_CHECKSUM)) { if (vallen != sizeof(int)) RETURN(-EINVAL); exp-&gt;exp_obd-&gt;u.cli.cl_checksum = (*(int *)val) ? 1 : 0; RETURN(0); } ... } </pre> </div></div> <p> </p> <p>There are three ways to set checksum support in Lustre:</p> <ul class="alternate" type="square"> <li>configure: --enable-checksum or --disable-checksum</li> <li>mount option: -o checksum/nochecksum</li> <li>lctl set_param: osc.*.checksums</li> </ul> <p>My question is if "--disable-checksum" is specified when configuring, can we enable checksum again with "-o checksum" or osc.*.checksums=1? I think no.</p> <p>It looks like "<tt>configure &#45;&#45;enable&#45;checksum</tt>" (<tt>ENABLE&#95;CHECKSUM</tt>) only affects the default mount option in <tt>client&#95;obd&#95;setup()</tt> and <tt>ll&#95;sbi&#95;init()</tt>, but doesn't actually disable any of the code. That means it should be possible to enable checksums even if <tt>&#45;&#45;disable&#45;checksum</tt> is used. There does look like one minor bug in <tt>client&#95;obd&#95;setup()</tt> - the initialization of <tt>cl&#95;supp&#95;cksum&#95;types</tt> is not done if <tt>ENABLE&#95;CHECKSUM</tt> is unset, but it should probably always do that.</p> <p>I think the call to <tt>obd&#95;set&#95;info&#95;async()</tt> could be removed if the current setting matches the <tt>ENABLE&#95;CHECKSUM</tt> definition and there isn't a <tt>checksum</tt> or <tt>nochecksum</tt> mount option. Maybe something like the following, to only change the OSC checksum values if it has been changed from the default value:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> checksum = sbi-&gt;ll_flags &amp; LL_SBI_CHECKSUM; #ifdef ENABLE_CHECKSUM default_checksum = LL_SBI_CHECKSUM; #endif <span class="code-keyword">if</span> (checksum ^ default_checksum) { err = obd_set_info_async(NULL, sbi-&gt;ll_dt_exp, sizeof(KEY_CHECKSUM), KEY_CHECKSUM, sizeof(checksum), &amp;checksum, NULL); <span class="code-keyword">if</span> (err) { CERROR(<span class="code-quote">"%s: Set checksum failed: rc = %d\n"</span>, sbi-&gt;ll_dt_exp-&gt;exp_obd-&gt;obd_name, err); GOTO(out_root, err); } } </pre> </div></div> <p>Got, thanks!</p> <p>Emoly Liu (emoly.liu@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32095" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32095</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10906" title="checksums parameter not persistent after reboot" class="issue-link" data-issue-key="LU-10906"><del>LU-10906</del></a> checksum: enable/disable checksum correctly<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: afe8d04a2aee007a9df086b7a1d70096d75d3330</p> <p>Emoly Liu (emoly.liu@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32131" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32131</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10906" title="checksums parameter not persistent after reboot" class="issue-link" data-issue-key="LU-10906"><del>LU-10906</del></a> llite: rename llite checksum_pages to checksums<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 0949d2acc3887d4673a617f46b01b6771c8a8695</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/32095/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32095/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10906" title="checksums parameter not persistent after reboot" class="issue-link" data-issue-key="LU-10906"><del>LU-10906</del></a> checksum: enable/disable checksum correctly<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: e9b13cd1daf959b40e9ff5c810368c67e9491e06</p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/33222" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33222</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10906" title="checksums parameter not persistent after reboot" class="issue-link" data-issue-key="LU-10906"><del>LU-10906</del></a> llite: create checksums to replace checksum_pages<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 34df45679fd854b903467a259102b7104dc2bd55</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/33222/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33222/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10906" title="checksums parameter not persistent after reboot" class="issue-link" data-issue-key="LU-10906"><del>LU-10906</del></a> llite: create checksums to replace checksum_pages<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 123ee3cf96dd152c55f7e46042eba85cd23310a1</p> <p>Landed for 2.12</p> Related LU-8066 LU-11011 LU-11058 Development Rank 1|hzzvpj: Rank (Obsolete) 9223372036854775807 Severity