https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-685 Lustre <p>Hi,</p> <p>At CEA they have experienced at least 3 hangs of an MDS in the following context:</p> <ul> <li>most of the memory (&gt; 99%) is allocated by the kernel, caching inodes + attributes (system have 64 GB of memory)</li> </ul> <ul> <li>31 threads (over 32 cores) are busy waiting with the following backtrace (all are kiblns_sd_xx)</li> </ul> <p>_spin_lock<br/> kiblnd_pool_alloc_node<br/> kiblnd_get_idle_tx<br/> kiblnd_check_sends<br/> kiblnd_post_rx<br/> kiblnd_recv<br/> lnet_ni_recv<br/> lnet_recv_put<br/> lnet_parse<br/> kiblnd_handle_rx<br/> kiblnd_rx_complete<br/> kiblnd_complete<br/> kiblnd_scheduler<br/> kernel_thread</p> <ul> <li>1 thread (kiblnd_sd_11 in this case) is scheduled with the following bt</li> </ul> <p>schedule<br/> cfs_schedule<br/> kiblnd_pool_alloc_node<br/> kiblnd_get_idle_tx<br/> kiblnd_check_sends<br/> kiblnd_post_rx<br/> kiblnd_recv<br/> lnet_ni_recv<br/> lnet_recv_put<br/> lnet_parse<br/> kiblnd_handle_rx<br/> kiblnd_rx_complete<br/> kiblnd_complete<br/> kiblnd_scheduler<br/> kernel_thread</p> <ul> <li>1 thread (router_checker) is busy waiting with the following bt (the 32 th)</li> </ul> <p>spin_lock<br/> kiblnd_pool_alloc_node<br/> kiblnd_get_idle_tx<br/> kiblnd_send<br/> lnet_ni_send<br/> lnet_send<br/> LNetGet<br/> lnet_router_checker<br/> kernel_thread</p> <ul> <li>1 thread (kiblnd_connd) is scheduled with the following bt (the one owning the lock?)</li> </ul> <p>schedule_timeout<br/> __down<br/> down<br/> ldlm_pools_shrink<br/> ldlm_polls_cli_shrink<br/> shrink_slab<br/> do_try_to_free_pages<br/> __alloc_pages_nodemask<br/> kmem_getpages<br/> fallback_alloc<br/> ____cache_alloc_node<br/> __kmalloc<br/> cfs_alloc<br/> kiblnd_alloc_pages<br/> kiblnd_create_tx_pool<br/> kiblnd_pool_alloc_node<br/> kiblnd_get_idle_tx<br/> kiblnd_check_sends<br/> kiblnd_check_conns<br/> kiblnd_connd<br/> kernel_kthread</p> <p>So, havind #cpu threads busy waiting on a spinlock is not ideal, especially when the owner seems to be scheduled awaiting for something else to happen. At the moment it is not possible to verify if the lock is the same for all threads because the crash dump is not accessible.</p> <p>Trying to workaround this issue, we made a patch that allows choosing how many kiblnd threads are started, via a kernel module option. We set this number to 24 (for 32 cores).<br/> The issue reproduced, this time having 24 threads spinning on a spinlock, 4 kswapd waiting to acquire a lock in lu_cache_shrink, most of the others threads stucks in congestion_wait.</p> <p>Cheers,<br/> Sebastien.</p> LU-685

Wide busy lock in kiblnd_pool_alloc_node

Bug Major Resolved Fixed Lai Siyao Sebastien Buisson Wed, 14 Sep 2011 11:00:01 +0000 Fri, 4 May 2012 09:03:24 +0000 Mon, 12 Dec 2011 12:09:54 +0000 Lustre 2.2.0 Lustre 2.1.2 0 8 <p>Well, we made some progress on this one. The root issue seems to be in the reclaim path which doesn't work. When we are in bad shape, forcing a reclaim (echo 3 &gt; /proc/sys/vm/drop_caches) doesn't free memory. We have verified by activating lustre debug log that the number of objects doesn't change between lu_cache_shrink invocation.</p> <p>After more investigations, we have verified that the assertion in the comment of lu_site_purge is not verified "(less then ten)". By scanning the first 100000 nodes (over 22 millions !) we found about 1500 objects with a h-&gt;loh_ref &gt; 0 (the first 128 were in this stat, which explain why no nodes are selected during reclaim).</p> <p>Our tentative to workaround this issue is to move busy object found during the lu_site_purge scan at tail of the lru, but may be not having those nodes into the lru would be a better idea ? (Based on the expectation that the number of busy object we have is normal).</p> <p>Alex.</p> <p>Lai</p> <p>Can you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>I am not sure the inefficient lu_site_purge() will cause system hang; IMHO it should only make system load high and memory reclamation slow.</p> <p>I will commit a patch to put only non-referenced objects in lru list to speed up object reclamation. And you may verify whether it helps.</p> <p>Patch is on <a href="http://review.whamcloud.com/#change,1384" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,1384</a></p> <p>My understanding is not a performance problem of lu_site_purge but more a 'design' issue.<br/> Please correct me if I'm wrong, but my understanding is :</p> <ul class="alternate" type="square"> <li>the kernel vm manger call lu_cache_shrink with nr_to_scan set to SHRINK_BATCH (=128)</li> <li>then call lu_site_purge with nr set to (at most) nr_to_scan.</li> <li>lu_site_purge scan the nr first objects of the lru list, avoiding all object with a h-&gt;loh_ref &gt; 0</li> </ul> <p>Our problem is that the first nr object of the lru list have h-&gt;loh_ref &gt; 0, so l_site_purge doesn't select any objects for being freed. Since the vm didn't made progress, it return to the callback logic and as nothing moved forward, the nr first entry of the list are still busy and we are not any more able to free memory ... the callback process continue again and again. At this point the system is out of memory and stop answering.</p> <p>I haven't see any process to scan something else than the first SHRINK_BATCH entry of the list.</p> <p>Whatever, moving busy object out of the list should fix the problem.</p> <p>I definitely agree with Alexandre. I looked at this code some time ago and also found out that memory pressure handling on the lu_object_header slab is quite poor. Not only objects in-use are kept in the LRU, but we have one LRU list per hash bucket which are scanned in ascending order. As a result, if some buckets have a lot of objects in-use, we still waste time scanning all those objects sitting at the end of the list before moving on to the next bucket.</p> <p>While i would like very much to have in-use objects out of the LRU list, i wonder if a quick patch like this would help a bit:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>diff --git a/lustre/obdclass/lu_object.c b/lustre/obdclass/lu_object.c index 7b9a87c..60ac319 100644 --- a/lustre/obdclass/lu_object.c +++ b/lustre/obdclass/lu_object.c @@ -303,6 +303,14 @@ int lu_site_purge(const struct lu_env *env, struct lu_site *s, int nr) cfs_hash_bd_lock(s-&gt;ls_obj_hash, &amp;bd, 1); bkt = cfs_hash_bd_extra_get(s-&gt;ls_obj_hash, &amp;bd); + if (bkt-&gt;lsb_busy &gt;= cfs_hash_bd_count_get(&amp;bd)) { + /* all objects are busy, skip this bucket */ + cfs_hash_bd_unlock(s-&gt;ls_obj_hash, &amp;bd, 1); + continue; + } else if (count &gt; 0) + count = min_t(__u32, count, + cfs_hash_bd_count_get(&amp;bd) - bkt-&gt;lsb_busy); + cfs_list_for_each_entry_safe(h, temp, &amp;bkt-&gt;lsb_lru, loh_lru) { /* * Objects are sorted in lru order, and "busy" </pre> </div></div> <p>Hi Alexandre, looking in to lu_site_purge(), I think it will try best to reclaim 'nr' objects, not just scan 'nr' objects. Did I miss anything?</p> <p>In fact, when working on it, we thought about 3 possible ways to fix/work-around this issue :</p> <p> _ just/simply raise the "SHRINK_BATCH" hard-coded value used in shrink_slab() to invoke the shrinkers. Quite very basic solution is'nt-it !!</p> <p> _ switch the "nr" 0-test/decrease and the busy/loh_ref test before loop-back, this in order to allow "nr" not-busy objects reclaim. But thinking about the fact that the "busy" state of objects on the LRU-list may not be transcient, we thought that this may lead to the situation that all the busy objects will progressivelly become linked at the very beginning of the LRU, thus causing a very/too long search for the not-busy ones !!... What do you think ??</p> <p> _ this is why we thought that may be a better idea would be to still work/parse only "SHRINK_BATCH" objects but move each busy object encountered at the end of the LRU, to allow for non-busy objects to be found during the next search ...</p> <p>I don't have a clear idea about how many object with a ref &gt; 0 we can found, but based on our scanning of the 100000 first entry, we have already found more than 1500 in this state.<br/> Based on that, the first solution will be difficult to implement (which value), and the second will certainly drive us to a performance issue at some point (have to scan a lot of entry before finding some .</p> <p>The latest seems to be a better approach if we doesn't want to move those objects out of the lru, no ? a simple move_tail when we find a loh_ref &gt; 0 should help :</p> <ul class="alternate" type="square"> <li>if (cfs_atomic_read(&amp;h-&gt;loh_ref) &gt; 0)<br/> + if (cfs_atomic_read(&amp;h-&gt;loh_ref) &gt; 0) { + cfs_list_move_tail(&amp;h-&gt;loh_lru, &amp;s-&gt;ls_lru); continue; + } <p>What do you think ?</p></li> </ul> <p>Could you check out the patch I posted above? In that patch object will be placed in lru list when the last refcount is put, and be removed from lru list when it's referenced. In this way 'busy' object won't be scanned in reclamation. Since lru list is only used for object reclamation, this should be the most efficient.</p> <p>Well, in-use objects are supposed to live <b>near</b> the end of the list already (see lu_object_find_try()).<br/> As long as we assume that we <b>have</b> to free "nr / #buckets" objects per bucket, if some buckets have less free objects than that, we would end up scanning the whole list. That's why i thought the patch i posted above might help a bit.</p> <p>&gt; Could you check out the patch I posted above? In that patch object will be placed in lru list<br/> &gt; when the last refcount is put, and be removed from lru list when it's referenced</p> <p>That's great. This is really the only clean solution IMHO.</p> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=el5,ib_stack=ofa/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,el5,ofa #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=el5,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,el5,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=client,distro=el6,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,client,el6,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=server,distro=el6,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,server,el6,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=sles11,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,sles11,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=server,distro=el5,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,server,el5,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=el6,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,el6,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=ubuntu1004,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,ubuntu1004,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=server,distro=el5,ib_stack=ofa/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,server,el5,ofa #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=client,distro=el5,ib_stack=ofa/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,client,el5,ofa #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=client,distro=el5,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,client,el5,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=server,distro=el6,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,server,el6,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=server,distro=el5,ib_stack=inkernel/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,server,el5,inkernel #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=server,distro=el5,ib_stack=ofa/370/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,server,el5,ofa #370</a><br/> <a href="https://jira.whamcloud.com/browse/LU-685" title="Wide busy lock in kiblnd_pool_alloc_node" class="issue-link" data-issue-key="LU-685"><del>LU-685</del></a> obdclass: lu_object reclamation is inefficient (Revision 598996a12f5818190635e1f3a948c9e6a77297b5)</p> <p> Result = SUCCESS<br/> Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=598996a12f5818190635e1f3a948c9e6a77297b5" class="external-link" target="_blank" rel="nofollow noopener">598996a12f5818190635e1f3a948c9e6a77297b5</a><br/> Files : </p> <ul> <li>lustre/obdclass/lu_object.c</li> </ul> <p><a href="http://review.whamcloud.com/#change,2628" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,2628</a><br/> back port to b2_1</p> Development Rank 1|hzv81j: Rank (Obsolete) 4837 Severity