https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5107 Lustre <p>With the latest 2.5.1 snapshot when I attempt to bring up a file system I'm seeing the following bug on the MDS during the MDT mount. Because of this I can' currently mount a 2.5 file system for testing.</p> <p>May 27 16:55:19 tick-dne-mds1 kernel: [ 546.512335] LustreError: 13869:0:(osp_dev.c:864:osp_prepare_fid_client()) ASSERTION( osp-&gt;opd<br/> _obd-&gt;u.cli.cl_seq != ((void *)0) ) failed:<br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.548335] LustreError: 13869:0:(osp_dev.c:864:osp_prepare_fid_client()) LBUG<br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.569232] Pid: 13869, comm: ptlrpcd_rcv<br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.579503]<br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.579503] Call Trace:<br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.598249] <span class="error">&#91;&lt;ffffffffa05f3895&gt;&#93;</span> libcfs_debug_dumpstack+0x55/0x80 <span class="error">&#91;libcfs&#93;</span><br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.618857] <span class="error">&#91;&lt;ffffffffa05f3e97&gt;&#93;</span> lbug_with_loc+0x47/0xb0 <span class="error">&#91;libcfs&#93;</span><br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.638835] <span class="error">&#91;&lt;ffffffffa108ee34&gt;&#93;</span> osp_import_event+0x3d4/0x410 <span class="error">&#91;osp&#93;</span><br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.659079] <span class="error">&#91;&lt;ffffffffa09207cc&gt;&#93;</span> ptlrpc_activate_import+0x12c/0x270 <span class="error">&#91;ptlrpc&#93;</span><br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.688094] <span class="error">&#91;&lt;ffffffffa0923502&gt;&#93;</span> ptlrpc_connect_interpret+0x1912/0x2160 <span class="error">&#91;ptlrpc&#93;</span><br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.708910] <span class="error">&#91;&lt;ffffffffa08f894c&gt;&#93;</span> ptlrpc_check_set+0x2bc/0x1b50 <span class="error">&#91;ptlrpc&#93;</span><br/> May 27 16:55:19 tick-dne-mds1 kernel: [ 546.729238] <span class="error">&#91;&lt;ffffffffa0924cab&gt;&#93;</span> ptlrpcd_check+0x53b/0x560 <span class="error">&#91;ptlrpc&#93;</span><br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.749421] <span class="error">&#91;&lt;ffffffffa09251cb&gt;&#93;</span> ptlrpcd+0x20b/0x370 <span class="error">&#91;ptlrpc&#93;</span><br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.769286] <span class="error">&#91;&lt;ffffffff81061d00&gt;&#93;</span> ? default_wake_function+0x0/0x20<br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.789361] <span class="error">&#91;&lt;ffffffffa0924fc0&gt;&#93;</span> ? ptlrpcd+0x0/0x370 <span class="error">&#91;ptlrpc&#93;</span><br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.809414] <span class="error">&#91;&lt;ffffffff8109ab56&gt;&#93;</span> kthread+0x96/0xa0<br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.828800] <span class="error">&#91;&lt;ffffffff8100c20a&gt;&#93;</span> child_rip+0xa/0x20<br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.848257] <span class="error">&#91;&lt;ffffffff8109aac0&gt;&#93;</span> ? kthread+0x0/0xa0<br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.859345] <span class="error">&#91;&lt;ffffffff8100c200&gt;&#93;</span> ? child_rip+0x0/0x20<br/> May 27 16:55:20 tick-dne-mds1 kernel: [ 546.879015]</p> MDS server LU-5107

MDS oops during mount with latest lustre 2.5.1 snapshot

Bug Blocker Resolved Fixed Di Wang James A Simmons Tue, 27 May 2014 21:02:53 +0000 Fri, 30 May 2014 19:48:56 +0000 Fri, 30 May 2014 19:48:56 +0000 Lustre 2.5.1 Lustre 2.5.2 0 5 <p>Di, </p> <p>Would you please comment on this ticket?</p> <p>Thank you,<br/> James</p> <p>James,</p> <p>Did you setup lustre with single MDT or DNE? Are there any other console error message? Could you tell me which build are you using? It is a new formatted FS? Do you have the dump log for this LBUG?</p> <p>Thank you.<br/> WangDi</p> <p>I tried a build with a few extra patches. Then I tried the tip of b2_5 and it was the same problem. Yes it is a DNE setup with 3 MDS servers. When I encountered this error I was using a already formatted 2.5 file system. I later reformatted to make sure that was not the issue but the MDS oops was still there. I found that reverting <a href="https://jira.whamcloud.com/browse/LU-4413" title="Test failure on test suite conf-sanity, subtest test_56" class="issue-link" data-issue-key="LU-4413"><del>LU-4413</del></a> appears to make the problem go away.<br/> I don't think reverting that patch is the solution. I have placed the dmesg log and vmcore dump at ftp.whamcloud.com/uploads/<a href="https://jira.whamcloud.com/browse/LU-5107" title="MDS oops during mount with latest lustre 2.5.1 snapshot" class="issue-link" data-issue-key="LU-5107"><del>LU-5107</del></a>.</p> <p>James, there are two patches on <a href="https://jira.whamcloud.com/browse/LU-4413" title="Test failure on test suite conf-sanity, subtest test_56" class="issue-link" data-issue-key="LU-4413"><del>LU-4413</del></a>:</p> <ul class="alternate" type="square"> <li><a href="http://review.whamcloud.com/8997" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8997</a> - osp: move seq allocation out of osp_import_event</li> <li><a href="http://review.whamcloud.com/8996" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8996</a> - ptlrpc: don't try to recover no_recov connection</li> </ul> <p>Which one did you revert to fix the problem?</p> <p>I reverted patch 8997.</p> <p>Hmm, there are some problems for 8997 when port it to 2.5. Since we do not need OSP(for MDT) to allocate FID, so osp_prepare_fid_client(d) needs to be moved after if (d-&gt;opd_connect_mdt) check in osp_import_event.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>diff --git a/lustre/osp/osp_dev.c b/lustre/osp/osp_dev.c index a4a2f90..15f2ec0 100644 --- a/lustre/osp/osp_dev.c +++ b/lustre/osp/osp_dev.c @@ -1053,15 +1053,16 @@ static int osp_import_event(struct obd_device *obd, struct obd_import *imp, case IMP_EVENT_ACTIVE: d-&gt;opd_imp_active = 1; - if (osp_prepare_fid_client(d) != 0) - break; - if (d-&gt;opd_got_disconnected) d-&gt;opd_new_connection = 1; d-&gt;opd_imp_connected = 1; d-&gt;opd_imp_seen_connected = 1; if (d-&gt;opd_connect_mdt) break; + + if (osp_prepare_fid_client(d) != 0) + break; + wake_up(&amp;d-&gt;opd_pre_waitq); __osp_sync_check_for_work(d); CDEBUG(D_HA, "got connected\n"); </pre> </div></div> <p>probably fix the problem, I will cook a patch.</p> <p><a href="http://review.whamcloud.com/10476" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10476</a></p> <p>The patch appears to have resolved the issue. Thank you.</p> <p>Problem was caused by backport of patch <a href="http://review.whamcloud.com/9875" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9875</a> to b2_5.</p> <p>Landed for 2.5.2. As I understand it, this issue only affected b2_5 so is not needed on other branches</p> Related LU-4413 Development Rank 1|hzwn53: Rank (Obsolete) 14091 Severity