https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-662 Lustre <p>After the clean upgrading, while running metabench test, client-13 hit the following LBUG:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Lustre: DEBUG MARKER: == parallel-scale test metabench: metabench == 23:53:25 (1315292005) LustreError: 21857:0:(fid_request.c:213:seq_client_alloc_seq()) ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed LustreError: 21857:0:(fid_request.c:213:seq_client_alloc_seq()) LBUG Pid: 21857, comm: metabench Call Trace: [&lt;ffffffff887b15f1&gt;] libcfs_debug_dumpstack+0x51/0x60 [libcfs] [&lt;ffffffff887b1b2a&gt;] lbug_with_loc+0x7a/0xd0 [libcfs] [&lt;ffffffff887bcde0&gt;] cfs_tracefile_init+0x0/0x10a [libcfs] [&lt;ffffffff88ab9cad&gt;] seq_client_alloc_fid+0x4bd/0x810 [fid] [&lt;ffffffff8001a870&gt;] vsnprintf+0x3f8/0x627 [&lt;ffffffff8008e430&gt;] default_wake_function+0x0/0xe [&lt;ffffffff887bcaeb&gt;] libcfs_debug_vmsg2+0x75b/0x9f0 [libcfs] [&lt;ffffffff88b0bced&gt;] mdc_fid_alloc+0x7d/0x100 [mdc] [&lt;ffffffff88b1df45&gt;] mdc_intent_lock+0x385/0x610 [mdc] [&lt;ffffffff88d1748c&gt;] ll_i2gids+0x6c/0xa0 [lustre] [&lt;ffffffff88d177e0&gt;] ll_md_blocking_ast+0x0/0x580 [lustre] [&lt;ffffffff88971cd0&gt;] ldlm_completion_ast+0x0/0x6a0 [ptlrpc] [&lt;ffffffff88d17038&gt;] ll_lookup_it+0x8c8/0xc90 [lustre] [&lt;ffffffff88d177e0&gt;] ll_md_blocking_ast+0x0/0x580 [lustre] [&lt;ffffffff88d18166&gt;] ll_lookup_nd+0x1e6/0x400 [lustre] [&lt;ffffffff80022a76&gt;] d_alloc+0x174/0x1a9 [&lt;ffffffff800370b8&gt;] __lookup_hash+0x10b/0x12f [&lt;ffffffff8001b227&gt;] open_namei+0xf2/0x712 [&lt;ffffffff8002768a&gt;] do_filp_open+0x1c/0x38 [&lt;ffffffff8001a061&gt;] do_sys_open+0x44/0xbe [&lt;ffffffff8005d28d&gt;] tracesys+0xd5/0xe0 Kernel panic - not syncing: LBUG </pre> </div></div> <p>Maloo report: <a href="https://maloo.whamcloud.com/test_sets/f2edb6ae-d863-11e0-8d02-52540025f9af" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/f2edb6ae-d863-11e0-8d02-52540025f9af</a></p> <br/> Old Lustre Versions: 1.8.5 and 1.8.6-wc1<br/> Lustre 1.8.6-wc1 Build: <a href="http://newbuild.whamcloud.com/job/lustre-b1_8/100/">http://newbuild.whamcloud.com/job/lustre-b1_8/100/</a><br/> <br/> New Lustre Version: 2.1.0<br/> Lustre 2.1.0 Build: <a href="http://newbuild.whamcloud.com/job/lustre-master/276/">http://newbuild.whamcloud.com/job/lustre-master/276/</a><br/> Network: TCP (1GigE)<br/> <br/> Clean upgrading (Lustre servers and clients were upgraded all at once) from Lustre 1.8.5 and 1.8.6-wc1 to Lustre 2.1.0 under the following configuration:<br/> OSS1: RHEL5/x86_64 upgrade from 1.8.6-wc1 to 2.1.0<br/> OSS2: RHEL5/x86_64 upgrade from 1.8.5 to 2.1.0<br/> MDS: RHEL5/x86_64 upgrade from 1.8.6-wc1 to 2.1.0<br/> Client1: RHEL6/x86_64 upgrade from 1.8.6-wc1 to 2.1.0<br/> Client2: RHEL5/x86_64 upgrade from 1.8.5 to 2.1.0<br/> <br/> Test nodes:<br/> OSS1: fat-amd-2<br/> OSS2: fat-amd-3<br/> MDS: fat-amd-1<br/> Client1: client-12<br/> Client2: client-13<br/> LU-662

1.8<->2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&seq->lcs_space)) failed

Bug Blocker Resolved Fixed Hongchao Zhang Jian Yu Tue, 6 Sep 2011 05:18:42 +0000 Fri, 6 Apr 2012 20:15:23 +0000 Wed, 14 Sep 2011 07:43:49 +0000 Lustre 2.1.0 Lustre 2.1.0 0 2 <p>HongChao</p> <p>Can you please look into this one as your top priority?</p> <p>Thanks</p> <p>Peter</p> <p>there is two possible cases to cause this issue,<br/> 1, there is a bug in "seq_client_proc_write_width"<br/> ...<br/> cfs_down(&amp;seq-&gt;lcs_sem);</p> <p> rc = lprocfs_write_helper(buffer, count, &amp;val);<br/> if (rc) &lt;------- there should add a cfs_up(&amp;seq-&gt;lcs_sem) to release the semaphore <br/> RETURN(rc);<br/> ...</p> <p>2, "seq_client_flush" conflicts with "seq_client_alloc_seq", which updates "lu_client_seq-&gt;lcs_space" without holding<br/> the "lcs_sem", and "seq_client_flush"(called in "mdc_import_event" for event "IMP_EVENT_INACTIVE") could reinitialize <br/> it at the same time.</p> <p>Hi HongChao - </p> <p>Please provide an update today on your thoughts on this so we can determine blocker status and decide whether we are ready to cut RC2 tomorrow.</p> <p>Thanks,</p> <p>Jessica</p> <p>patch is at <a href="http://review.whamcloud.com/#change,1364" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,1364</a></p> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=el5,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,el5,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=client,distro=el6,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,client,el6,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=server,distro=el5,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,server,el5,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/fid_request.c</li> <li>lustre/fid/lproc_fid.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=server,distro=el6,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,server,el6,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/fid_request.c</li> <li>lustre/fid/lproc_fid.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=sles11,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,sles11,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/fid_request.c</li> <li>lustre/fid/lproc_fid.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=el6,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,el6,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=el5,ib_stack=ofa/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,el5,ofa #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=client,distro=ubuntu1004,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,client,ubuntu1004,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=x86_64,build_type=server,distro=el5,ib_stack=ofa/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » x86_64,server,el5,ofa #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=server,distro=el6,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,server,el6,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=server,distro=el5,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,server,el5,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=client,distro=el5,ib_stack=ofa/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,client,el5,ofa #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=client,distro=el5,ib_stack=inkernel/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,client,el5,inkernel #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Integrated in <span class="image-wrap" style=""><img src="http://build.whamcloud.com/images/16x16/blue.png" style="border: 0px solid black" /></span> <a href="http://build.whamcloud.com/job/lustre-master/./arch=i686,build_type=server,distro=el5,ib_stack=ofa/278/" class="external-link" target="_blank" rel="nofollow noopener">lustre-master » i686,server,el5,ofa #278</a><br/> <a href="https://jira.whamcloud.com/browse/LU-662" title="1.8&lt;-&gt;2.1 interop: LBUG: ASSERTION(!range_is_exhausted(&amp;seq-&gt;lcs_space)) failed" class="issue-link" data-issue-key="LU-662"><del>LU-662</del></a> fix conflict between seq_client_flush and seq_client_alloc_fid</p> <p>Oleg Drokin : <a href="http://git.whamcloud.com/gitweb?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master&amp;a=commit&amp;h=d1feb5c774d4690a4d4c4828d734a2604438f923" class="external-link" target="_blank" rel="nofollow noopener">d1feb5c774d4690a4d4c4828d734a2604438f923</a><br/> Files : </p> <ul> <li>lustre/fid/lproc_fid.c</li> <li>lustre/fid/fid_request.c</li> </ul> <p>Landed for 2.1</p> <p>Just FYI, I think our BG/P system running our 1.8 client is hitting this. So it should go away when we upgrade to 2.1.1+ on our servers.</p> Development Rank 1|hzv8gf: Rank (Obsolete) 4904 Severity