https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2726 Lustre <p>Probably caused by landing of <a href="http://review.whamcloud.com/3246" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3246</a> I now get this crash:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[ 5006.818563] LustreError: 138-a: lustre-OST0001: A client on nid 0@lo was evicted due to a lock blocking callback time out: rc -107 [ 5006.820054] LustreError: Skipped 1 previous similar message [ 5006.820979] LustreError: 3302:0:(lvfs_lib.c:131:lprocfs_counter_sub()) ASSERTION( (!((stats-&gt;ls_flags &amp; LPROCFS_STATS_FLAG_IRQ_SAFE) == 0) || (!(((current_thread_info()-&gt;preempt_count) &amp; ((((1UL &lt;&lt; (10))-1) &lt;&lt; ((0 + 8) + 8)) | (((1UL &lt;&lt; (8))-1) &lt;&lt; (0 + 8)) | (((1UL &lt;&lt; (1))-1) &lt;&lt; (((0 + 8) + 8) + 10))))))) ) failed: [ 5006.822543] LustreError: 3302:0:(lvfs_lib.c:131:lprocfs_counter_sub()) LBUG [ 5006.822805] Kernel panic - not syncing: LBUG in interrupt. [ 5006.822806] [ 5006.823205] Pid: 3302, comm: ldlm_elt Not tainted 2.6.32-debug #6 [ 5006.823447] Call Trace: [ 5006.823635] [&lt;ffffffff814f75e4&gt;] ? panic+0xa0/0x168 [ 5006.823872] [&lt;ffffffffa0449f5d&gt;] ? lbug_with_loc+0x8d/0xb0 [libcfs] [ 5006.824123] [&lt;ffffffffa05a3027&gt;] ? lprocfs_counter_sub+0x177/0x190 [lvfs] [ 5006.824403] [&lt;ffffffffa07414ff&gt;] ? ldlm_lock_put+0xcf/0x540 [ptlrpc] [ 5006.824698] [&lt;ffffffffa0767fe0&gt;] ? expired_lock_main+0x660/0x830 [ptlrpc] [ 5006.825038] [&lt;ffffffff81057d60&gt;] ? default_wake_function+0x0/0x20 [ 5006.825301] [&lt;ffffffffa0767980&gt;] ? expired_lock_main+0x0/0x830 [ptlrpc] [ 5006.825555] [&lt;ffffffff8100c14a&gt;] ? child_rip+0xa/0x20 [ 5006.825802] [&lt;ffffffffa0767980&gt;] ? expired_lock_main+0x0/0x830 [ptlrpc] [ 5006.826071] [&lt;ffffffffa0767980&gt;] ? expired_lock_main+0x0/0x830 [ptlrpc] [ 5006.826325] [&lt;ffffffff8100c140&gt;] ? child_rip+0x0/0x20 </pre> </div></div> <p>I have a crashdump in /exports/crashdumps/192.168.10.223-2013-01-31-14\:11\:49/vmcore</p> LU-2726

lprocfs_counter_sub()) ASSERTION( (!((stats->ls_flags & LPROCFS_STATS_FLAG_IRQ_SAFE) == 0) || (!(((current_thread_info()->preempt_count) &... ) failed

Bug Blocker Resolved Fixed Zhenyu Xu Oleg Drokin HB Thu, 31 Jan 2013 14:19:10 +0000 Fri, 8 Feb 2013 14:08:23 +0000 Fri, 8 Feb 2013 14:08:23 +0000 Lustre 2.4.0 Lustre 2.4.0 0 6 <p>Please see <a href="http://review.whamcloud.com/5236" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5236</a>.</p> <p>patch tracking <a href="http://review.whamcloud.com/5237" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5237</a></p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedHeader panelHeader" style="border-bottom-width: 1px;"><b>commit message</b></div><div class="preformattedContent panelContent"> <pre>LU-2726 procfs: Invalid assertion for counter_{add,sub} lprocfs_stats::lc_sum_irq is only used for obd_memory for accurate memory usage counting, other procfs stats counters do not need it, even though they can be used in RCU callbacks. </pre> </div></div> <p>Oleg is checking to see if Change, 5237 corrects the problem and can be landed.</p> <p>I've got another LBUG with this patch:</p> <p><a href="https://maloo.whamcloud.com/test_sessions/6c96e10e-716c-11e2-9241-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sessions/6c96e10e-716c-11e2-9241-52540035b04c</a></p> <p>MDS console:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">09:33:10:LNetError: 23499:0:(lib-lnet.h:489:lnet_me_alloc()) ASSERTION( !(((current_thread_info()-&gt;preempt_count) &amp; ((((1UL &lt;&lt; (10))-1) &lt;&lt; ((0 + 8) + 8)) | (((1UL &lt;&lt; (8))-1) &lt;&lt; (0 + 8)) | (((1UL &lt;&lt; (1))-1) &lt;&lt; (((0 + 8) + 8) + 10))))) || (((sizeof(*me))) &lt;= (2 &lt;&lt; 12) &amp;&amp; (((CFS_ALLOC_IO)) &amp; CFS_ALLOC_ATOMIC)) != 0 ) failed: 09:33:10:LNetError: 23499:0:(lib-lnet.h:489:lnet_me_alloc()) LBUG 09:33:10:Kernel panic - not syncing: LBUG in interrupt. 09:33:10: 09:33:10:Pid: 23499, comm: mdt00_003 Not tainted 2.6.32-279.19.1.el6_lustre.gc7121e9.x86_64 #1 09:33:10:Call Trace: 09:33:10: [&lt;ffffffff814e9811&gt;] ? panic+0xa0/0x168 09:33:10: [&lt;ffffffffa0448edd&gt;] ? lbug_with_loc+0x8d/0xb0 [libcfs] 09:33:10: [&lt;ffffffffa04cdffe&gt;] ? lnet_me_alloc+0x1ce/0x1d0 [lnet] 09:33:10: [&lt;ffffffffa04ce3a8&gt;] ? LNetMEAttach+0x98/0x220 [lnet] 09:33:10: [&lt;ffffffffa070d1e2&gt;] ? ptlrpc_register_rqbd+0x82/0x390 [ptlrpc] 09:33:10: [&lt;ffffffffa04495de&gt;] ? cfs_timer_arm+0xe/0x10 [libcfs] 09:33:10: [&lt;ffffffffa045adcf&gt;] ? lc_watchdog_touch+0x6f/0x170 [libcfs] 09:33:10: [&lt;ffffffffa071a129&gt;] ? ptlrpc_wait_event+0xa9/0x290 [ptlrpc] 09:33:10: [&lt;ffffffffa04592e1&gt;] ? libcfs_debug_msg+0x41/0x50 [libcfs] 09:33:10: [&lt;ffffffffa071adc5&gt;] ? ptlrpc_server_post_idle_rqbds+0x75/0xe0 [ptlrpc] 09:33:10: [&lt;ffffffffa07240a2&gt;] ? ptlrpc_main+0xb92/0x19e0 [ptlrpc] 09:33:10: [&lt;ffffffffa0723510&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] 09:33:10: [&lt;ffffffff8100c0ca&gt;] ? child_rip+0xa/0x20 09:33:10: [&lt;ffffffffa0723510&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] 09:33:10: [&lt;ffffffffa0723510&gt;] ? ptlrpc_main+0x0/0x19e0 [ptlrpc] 09:33:10: [&lt;ffffffff8100c0c0&gt;] ? child_rip+0x0/0x20 </pre> </div></div> <p>this is another issue, deserves a new jira ticket.</p> <p>lnet_me_alloc-&gt;LIBCFS_ALLOC-&gt;LIBCFS_ALLOC_GFP-&gt;LIBCFS_ALLOC_PRE</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">#define LIBCFS_ALLOC_PRE(size, mask) \ <span class="code-keyword">do</span> { \ LASSERT(!cfs_in_interrupt() || \ ((size) &lt;= LIBCFS_VMALLOC_SIZE &amp;&amp; \ ((mask) &amp; CFS_ALLOC_ATOMIC)) != 0); \ } <span class="code-keyword">while</span> (0) </pre> </div></div> <p>I think it's alloc GFP mask issue.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> /** * <span class="code-keyword">default</span> allocator */ #define LIBCFS_ALLOC(ptr, size) \ LIBCFS_ALLOC_GFP(ptr, size, CFS_ALLOC_IO) /** * non-sleeping allocator */ #define LIBCFS_ALLOC_ATOMIC(ptr, size) \ LIBCFS_ALLOC_GFP(ptr, size, CFS_ALLOC_ATOMIC) </pre> </div></div> <p>Liang, could you take it a look?</p> <p>Hi Mike, I think the issue reported by you is because a defect in your patch, <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/>, please check my comment in your patch.</p> <p>Liang, thanks!</p> <p>Patch landed, issue no longer happens in my testing</p> Development Rank 1|hzvidb: Rank (Obsolete) 6626 Severity