https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2127 Lustre <p>Some examples, taken for convenience from <a href="https://jira.whamcloud.com/browse/LU-1153" title="Client Unresponsive" class="issue-link" data-issue-key="LU-1153"><del>LU-1153</del></a> logs but which are very frequent on T100 site :<br/> =================================================================================================</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Feb 28 09:37:28 compute-01-32 kernel: LustreError: 2567:0:(osc_request.c:797:osc_announce_cached()) dirty 2028 - 2028 &gt; system dirty_max 647168 Feb 28 09:37:40 compute-01-32 kernel: LustreError: 2568:0:(osc_request.c:797:osc_announce_cached()) dirty 2040 - 2040 &gt; system dirty_max 647168 Feb 28 09:37:46 compute-01-32 kernel: LustreError: 2565:0:(osc_request.c:797:osc_announce_cached()) dirty 1954 - 1954 &gt; system dirty_max 647168 Feb 28 09:37:47 compute-01-32 kernel: LustreError: 2561:0:(osc_request.c:797:osc_announce_cached()) dirty 1975 - 1976 &gt; system dirty_max 647168 Feb 28 09:37:50 compute-01-32 kernel: LustreError: 2566:0:(osc_request.c:797:osc_announce_cached()) dirty 2001 - 2002 &gt; system dirty_max 647168 Feb 28 09:38:09 compute-01-32 kernel: LustreError: 2567:0:(osc_request.c:797:osc_announce_cached()) dirty 1946 - 1946 &gt; system dirty_max 647168 Feb 28 09:39:20 compute-01-32 kernel: LustreError: 2615:0:(osc_request.c:797:osc_announce_cached()) dirty 2017 - 2017 &gt; system dirty_max 647168 Feb 28 09:39:47 compute-01-32 kernel: LustreError: 2615:0:(osc_request.c:797:osc_announce_cached()) dirty 1847 - 1848 &gt; system dirty_max 647168 </pre> </div></div> <p>=================================================================================================</p> <p>this comes from the following piece of code :<br/> =============================================</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">. } <span class="code-keyword">else</span> <span class="code-keyword">if</span> (cfs_atomic_read(&amp;obd_dirty_pages) - cfs_atomic_read(&amp;obd_dirty_transit_pages) &gt; obd_max_dirty_pages + 1){ /* The cfs_atomic_read() allowing the cfs_atomic_inc() are * not covered by a lock thus they may safely race and trip * <span class="code-keyword">this</span> CERROR() unless we add in a small fudge factor (+1). */ CERROR(<span class="code-quote">"dirty %d - %d &gt; system dirty_max %d\n"</span>, cfs_atomic_read(&amp;obd_dirty_pages), cfs_atomic_read(&amp;obd_dirty_transit_pages), obd_max_dirty_pages); </pre> </div></div> <p>=============================================</p> <p>where both obd_dirty_pages/obd_dirty_transit_pages are "likely to change" signed ints and obd_max_dirty_pages unsigned which an easily explain the wrong error msgs ...</p> <p>This is still true in current master and, if I am right and there is no tricky stuff I may have missed there ..., the following patch should fix :<br/> ==================================================================================================================================================</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">diff --git a/lustre/osc/osc_request.c b/lustre/osc/osc_request.c index af60e4b..5f75329 100644 --- a/lustre/osc/osc_request.c +++ b/lustre/osc/osc_request.c @@ -788,6 +788,7 @@ <span class="code-keyword">static</span> void osc_announce_cached(struct client_obd *cli, struct obdo *oa, <span class="code-object">long</span> writing_bytes) { obd_flag bits = OBD_MD_FLBLOCKS|OBD_MD_FLGRANT; + <span class="code-object">int</span> diff; LASSERT(!(oa-&gt;o_valid &amp; bits)); @@ -798,9 +799,9 @@ <span class="code-keyword">static</span> void osc_announce_cached(struct client_obd *cli, struct obdo *oa, CERROR(<span class="code-quote">"dirty %lu - %lu &gt; dirty_max %lu\n"</span>, cli-&gt;cl_dirty, cli-&gt;cl_dirty_transit, cli-&gt;cl_dirty_max); oa-&gt;o_undirty = 0; - } <span class="code-keyword">else</span> <span class="code-keyword">if</span> (cfs_atomic_read(&amp;obd_dirty_pages) - - cfs_atomic_read(&amp;obd_dirty_transit_pages) &gt; - obd_max_dirty_pages + 1){ + } <span class="code-keyword">else</span> <span class="code-keyword">if</span> (((diff = (cfs_atomic_read(&amp;obd_dirty_pages) - + cfs_atomic_read(&amp;obd_dirty_transit_pages))) &gt; + obd_max_dirty_pages + 1) &amp;&amp; (diff &gt; 0)){ /* The cfs_atomic_read() allowing the cfs_atomic_inc() are * not covered by a lock thus they may safely race and trip * <span class="code-keyword">this</span> CERROR() unless we add in a small fudge factor (+1). */ </pre> </div></div> <p>==================================================================================================================================================</p> LU-2127

Confusing messages from osc_announce_cached() caused by unchecked signed/unsigned comparison

Bug Minor Resolved Fixed Keith Mannthey Alexandre Louvet mq213 Tue, 9 Oct 2012 12:40:51 +0000 Mon, 25 Feb 2013 11:10:54 +0000 Mon, 25 Feb 2013 11:10:54 +0000 Lustre 2.4.0 0 7 <p>Oops, sorry for the wrong formating, don't know what happen exactly but seems the starting +/- characters of each patch line did put the tool/inputs into some trouble ...</p> <p>Anyway, I can push my patch to master but I just want to get some feedback before ...</p> <p>Bruno,<br/> thanks for the bug report and proposed fix.</p> <p>The Lustre coding style discourages the use of assignments inside conditionals, because it can lead to hard-to-find bugs in the code. Since we always expect the "diff" calculation to be needed, except in the very rare case that the first "if" condition is true, then it would be fine to compute "diff" before all of the checks.</p> <p>I think it would also be possible to avoid this error my casting <tt>(long)obd_max_dirty_pages</tt> so that it is doing a signed comparison. This avoids adding a second check into the code and an extra variable on the stack and I think makes it more clear.</p> <p>Also, I think while this code is being changed that a few more cleanups could be done here:</p> <ul class="alternate" type="square"> <li>add space between '){' at the end of the "else if" line in question</li> <li>the first 3 cases should be annotated with unlikely(), since they are error cases that shouldn't happen</li> <li>use tabs instead of spaces for indentation for modified lines and neighboring lines</li> <li>wrap long line (max_in_flight) at 80 characters</li> </ul> <p>Bruno</p> <p>Thanks again for the suggested change. Could you please submit the next version of the code into gerrit so we can test and land it?</p> <p>Keith</p> <p>Could you please help see this through to landing?</p> <p>Thanks</p> <p>Peter</p> <p>Thank's for these hints Andreas !!</p> <p>My 1st patch version was definitelly to request feedback on the best/preferred way to fix <br/> and BTW later running the "build/checkpatch.pl" script was scheduled in order to ensure <br/> beautifying the patch. But seems that the osc_request.c module, at least the original <br/> osc_announce_cached() routine lines, did not follow the space/tab rules already...</p> <p>In fact, the real question I had in mind was use the (cast) for obd_max_dirty_pages or <br/> check that the difference did not become negative, because I personnaly don't like to use <br/> sign/size-casts because they always look tricky for me and that's why I proposed the 2nd way.</p> <p>But anyway, patch has been pushed to master as <a href="http://review.whamcloud.com/4246" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/4246</a></p> <p>I asked Andreas to review my patch for sure, but since it seems I need 2 reviewers, Keith can I ask you to review ??</p> <p>I see this is in 2.4.</p> <p>This also applies to earlier 2.x versions - It might be worth pushing the patch to those as well.</p> <p>Landed for 2.4 and yes this is a candidate for inclusion in an upcoming 2.1.x release</p> Development Rank 1|hzv9uf: Rank (Obsolete) 5129 Severity