https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3430 Lustre <p>Running SWL test with NRS policy 'orr' after 25 hours OSS had LBUG, there were multiple assertions during the initial stack dump:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2013-05-30 13:12:46 LustreError: 5770:0:(hash.c:546:cfs_hash_bd_del_locked()) ASSERTION( bd-&gt;bd_bucket-&gt;hsb_count &gt; 0 ) failed: 2013-05-30 13:12:46 LustreError: 5770:0:(hash.c:546:cfs_hash_bd_del_locked()) LBUG 2013-05-30 13:12:46 Pid: 5770, comm: ll_ost_io00_077 2013-05-30 13:12:46 2013-05-30 13:12:46 Call Trace: 2013-05-30 13:12:46 [&lt;ffffffffa04d1895&gt;] libcfs_debug_dumpstack+0x55/0x80 [libcfs] 2013-05-30 13:12:46 May 30 13:12:46 [&lt;ffffffffa04d1e97&gt;] lbug_with_loc+0x47/0xb0 [libcfs] 2013-05-30 13:12:46 hyperion-dit33 k [&lt;ffffffffa04e785a&gt;] cfs_hash_bd_del_locked+0xda/0x140 [libcfs] 2013-05-30 13:12:46 ernel: LustreErr [&lt;ffffffffa0a467e8&gt;] nrs_orr_hop_put_free+0x218/0x290 [ptlrpc] 2013-05-30 13:12:46 or: 5770:0:(hash [&lt;ffffffffa0a456d8&gt;] nrs_orr_res_put+0x28/0x60 [ptlrpc] 2013-05-30 13:12:46 .c:546:cfs_hash_ [&lt;ffffffffa0a3eb80&gt;] nrs_resource_put_safe+0x60/0xf0 [ptlrpc] 2013-05-30 13:12:46 bd_del_locked()) [&lt;ffffffffa0a3ec30&gt;] ptlrpc_nrs_req_finalize+0x20/0x30 [ptlrpc] 2013-05-30 13:12:46 ASSERTION( bd-&gt;bd_bucket-&gt;hsb_c [&lt;ffffffffa0a05a32&gt;] ptlrpc_server_finish_active_request+0x62/0x150 [ptlrpc] 2013-05-30 13:12:46 ount &gt; 0 ) faile [&lt;ffffffffa0a0c1a2&gt;] ptlrpc_server_handle_request+0x1b2/0xc60 [ptlrpc] 2013-05-30 13:12:46 d: 2013-05-30 13:12:46 May 30 13:12 [&lt;ffffffffa04d25de&gt;] ? cfs_timer_arm+0xe/0x10 [libcfs] 2013-05-30 13:12:46 :46 hyperion-dit [&lt;ffffffffa04e3d8f&gt;] ? lc_watchdog_touch+0x6f/0x170 [libcfs] 2013-05-30 13:12:46 33 kernel: Lustr [&lt;ffffffffa0a036e9&gt;] ? ptlrpc_wait_event+0xa9/0x290 [ptlrpc] 2013-05-30 13:12:46 eError: 5770:0:( [&lt;ffffffffa0a0d71e&gt;] ptlrpc_main+0xace/0x1700 [ptlrpc] 2013-05-30 13:12:46 hash.c:546:cfs_h [&lt;ffffffffa0a0cc50&gt;] ? ptlrpc_main+0x0/0x1700 [ptlrpc] 2013-05-30 13:12:46 ash_bd_del_locke [&lt;ffffffff8100c0ca&gt;] child_rip+0xa/0x20 2013-05-30 13:12:46 d()) LBUG 2013-05-30 13:12:46 [&lt;ffffffffa0a0cc50&gt;] ? ptlrpc_main+0x0/0x1700 [ptlrpc] 2013-05-30 13:12:46 [&lt;ffffffffa0a0cc50&gt;] ? ptlrpc_main+0x0/0x1700 [ptlrpc] 2013-05-30 13:12:46 [&lt;ffffffff8100c0c0&gt;] ? child_rip+0x0/0x20 2013-05-30 13:12:46 2013-05-30 13:12:46 Kernel panic - not syncing: LBUG 2013-05-30 13:12:46 Pid: 5770, comm: ll_ost_io00_077 Tainted: P --------------- 2.6.32-358.6.2.el6_lustre.g230b174.x86_64 #1 2013-05-30 13:12:46 Call Trace: 2013-05-30 13:12:46 [&lt;ffffffff8150d878&gt;] ? panic+0xa7/0x16f 2013-05-30 13:12:46 May 30 13:12:46 [&lt;ffffffffa04d1eeb&gt;] ? lbug_with_loc+0x9b/0xb0 [libcfs] 2013-05-30 13:12:46 hyperion-dit33 k [&lt;ffffffffa04e785a&gt;] ? cfs_hash_bd_del_locked+0xda/0x140 [libcfs] 2013-05-30 13:12:46 ernel: Kernel pa [&lt;ffffffffa0a467e8&gt;] ? nrs_orr_hop_put_free+0x218/0x290 [ptlrpc] 2013-05-30 13:12:46 nic - not syncin [&lt;ffffffffa0a456d8&gt;] ? nrs_orr_res_put+0x28/0x60 [ptlrpc] 2013-05-30 13:12:46 g: LBUG 2013-05-30 13:12:46 [&lt;ffffffffa0a3eb80&gt;] ? nrs_resource_put_safe+0x60/0xf0 [ptlrpc] 2013-05-30 13:12:46 [&lt;ffffffffa0a3ec30&gt;] ? ptlrpc_nrs_req_finalize+0x20/0x30 [ptlrpc] 2013-05-30 13:12:46 [&lt;ffffffffa0a05a32&gt;] ? ptlrpc_server_finish_active_request+0x62/0x150 [ptlrpc] 2013-05-30 13:12:46 [&lt;ffffffffa0a0c1a2&gt;] ? ptlrpc_server_handle_request+0x1b2/0xc60 [ptlrpc] 2013-05-30 13:12:46 [&lt;ffffffffa04d25de&gt;] ? cfs_timer_arm+0xe/0x10 [libcfs] 2013-05-30 13:12:46 [&lt;ffffffffa04e3d8f&gt;] ? lc_watchdog_touch+0x6f/0x170 [libcfs] 2013-05-30 13:12:46 [&lt;ffffffffa0a036e9&gt;] ? ptlrpc_wait_event+0xa9/0x290 [ptlrpc] 2013-05-30 13:12:46 [&lt;ffffffffa0a0d71e&gt;] ? ptlrpc_main+0xace/0x1700 [ptlrpc] 2013-05-30 13:12:47 [&lt;ffffffffa0a0cc50&gt;] ? ptlrpc_main+0x0/0x1700 [ptlrpc] 2013-05-30 13:12:47 [&lt;ffffffff8100c0ca&gt;] ? child_rip+0xa/0x20 2013-05-30 13:12:47 [&lt;ffffffffa0a0cc50&gt;] ? ptlrpc_main+0x0/0x1700 [ptlrpc] 2013-05-30 13:12:47 [&lt;ffffffffa0a0cc50&gt;] ? ptlrpc_main+0x0/0x1700 [ptlrpc] 2013-05-30 13:12:47 [&lt;ffffffff8100c0c0&gt;] ? child_rip+0x0/0x20 2013-05-30 13:12:47 Initializing cgroup subsys cpuset </pre> </div></div> LLNL/Hyperion LU-3430

SWL failure: (hash.c:546:cfs_hash_bd_del_locked()) ASSERTION( bd->bd_bucket->hsb_count > 0 ) failed:

Bug Critical Resolved Fixed Emoly Liu Cliff White patch Fri, 31 May 2013 15:48:48 +0000 Wed, 6 Nov 2013 15:24:32 +0000 Tue, 24 Sep 2013 17:33:51 +0000 Lustre 2.4.0 Lustre 2.5.0 Lustre 2.4.2 0 7 <p>i think a check for oo_ref is missing in nrs_orr_hop_put_free(); i will try to look at this issue today or tomorrow.</p> <p>Nikitas, any update on this problem?</p> <p>Hi Andreas,</p> <p>I have just been able to start working on this and other pending NRS issues, so I aim to have patches uploaded to Gerrit asap. </p> <p>patch for master is at <a href="http://review.whamcloud.com/#/c/7623" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/7623</a></p> <p>Emoly</p> <p>Can you please take care of this patch?</p> <p>Thanks</p> <p>Peter</p> <p>Cliff, patch is landed to master along with <a href="http://review.whamcloud.com/7708" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/7708</a> (<a href="https://jira.whamcloud.com/browse/LU-3978" title="Typos in NRS ORR/TRR policy completely invalidate scheduling behaviour" class="issue-link" data-issue-key="LU-3978"><del>LU-3978</del></a>). Can you please confirm that this fixes the problems with NRS ORR policy? Hopefully we will also get some performance improvement now, maybe more noticeably for ZFS...</p> <p>Patch is landed, closing bug.</p> <p>backport to b2_4: <a href="http://review.whamcloud.com/8162" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8162</a></p> Development Rank 1|hzvsf3: Rank (Obsolete) 8505 Severity