https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3142 Lustre <p>While running recovery-mds-scale test_failover_mds, dd operation failed on one of the client nodes as follows:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>2013-04-08 22:25:26: dd run starting + mkdir -p /mnt/lustre/d0.dd-client-32vm5.lab.whamcloud.com + /usr/bin/lfs setstripe -c -1 /mnt/lustre/d0.dd-client-32vm5.lab.whamcloud.com + cd /mnt/lustre/d0.dd-client-32vm5.lab.whamcloud.com ++ /usr/bin/lfs df /mnt/lustre/d0.dd-client-32vm5.lab.whamcloud.com + FREE_SPACE=12963076 + BLKS=2916692 + echo 'Free disk space is 12963076, 4k blocks to dd is 2916692' + load_pid=8739 + wait 8739 + dd bs=4k count=2916692 status=noxfer if=/dev/zero of=/mnt/lustre/d0.dd-client-32vm5.lab.whamcloud.com/dd-file dd: writing `/mnt/lustre/d0.dd-client-32vm5.lab.whamcloud.com/dd-file': Bad file descriptor 295176+0 records in 295175+0 records out + '[' 1 -eq 0 ']' ++ date '+%F %H:%M:%S' + echoerr '2013-04-08 22:27:28: dd failed' + echo '2013-04-08 22:27:28: dd failed' 2013-04-08 22:27:28: dd failed </pre> </div></div> <p>Maloo report: <a href="https://maloo.whamcloud.com/test_sets/68bce4aa-a1bb-11e2-bdac-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/68bce4aa-a1bb-11e2-bdac-52540035b04c</a></p> <br/> Lustre Branch: master<br/> Lustre Build: <a href="http://build.whamcloud.com/job/lustre-master/1381/">http://build.whamcloud.com/job/lustre-master/1381/</a><br/> Distro/Arch: RHEL6.3/x86_64<br/> Test Group: failover<br/> FAILURE_MODE=HARD<br/> LU-3142

recovery-mds-scale test_failover_mds: dd: writing `/mnt/lustre/d0.dd-client-32vm5.lab.whamcloud.com/dd-file': Bad file descriptor

Bug Blocker Resolved Fixed Hongchao Zhang Jian Yu LB Wed, 10 Apr 2013 15:13:52 +0000 Thu, 6 Jun 2013 12:50:13 +0000 Sat, 27 Apr 2013 21:04:19 +0000 Lustre 2.4.0 Lustre 2.4.0 0 7 <p>Are we able to pass any MDS failovers, or do they fail 100% of the time? It appears that this test failed immediately on the first MDS failover, but we don't have any useful logs from the MDS, so it is difficult to know why the OSTs were evicted.</p> <p>Hongchao</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>The recovery-*-scale tests on master branch have been blocked by <a href="https://jira.whamcloud.com/browse/LU-2008" title="After hardware reboot (using pm) the node cannot be accessed" class="issue-link" data-issue-key="LU-2008"><del>LU-2008</del></a>. After the issue was fixed 2 days ago, the hard failover tests were started being performed by autotest. I submitted <a href="http://review.whamcloud.com/6013" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/6013</a> to reproduce the issue.</p> <p>the logs in MDS doesn't contain any valid info about Lustre.</p> <p>the error "Bad file descriptor" (-EBADFD) is not a common error, there is only one place in Lustre (in ll_statahead_interpret),<br/> and in Linux, it's only in the following modules</p> <p>driver/<br/> isdn, net, macintosh, ieee1394, atm, media, usb<br/> fs/<br/> jfss2, ncpfs<br/> net/<br/> iucv, atm, 9p, bluetooth<br/> sound/<br/> core, drivers, usb</p> <p>then this error could come from driver modules, or trigger at user space.</p> <p>Is this actually -EBADF (which is a different error code)? Are there any messages about that in the console log? Are you sure that this was build 1381 (commit 49b06fba39e7fec26a0250ed37f04a620e349b5f) being tested? If it was a later build it might have been caused by commit <a href="http://review.whamcloud.com/5820" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5820</a>.</p> <blockquote><p>Is this actually -EBADF (which is a different error code)? Are there any messages about that in the console log? Are you sure that this was build 1381 (commit 49b06fba39e7fec26a0250ed37f04a620e349b5f) being tested? If it was a later build it might have been caused by commit <a href="http://review.whamcloud.com/5820" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5820</a>.</p></blockquote> <p>I did not find -EBADF(-9) or -EBADFD(-77) in the console logs. Due to TT-1107, the console logs were not gathered completely in the Maloo report. Please refer to the attached tarball. I'm sure this was build <a href="http://build.whamcloud.com/job/lustre-master/1381/" class="external-link" target="_blank" rel="nofollow noopener">http://build.whamcloud.com/job/lustre-master/1381/</a>.</p> <p>The debug patch in <a href="http://review.whamcloud.com/#change,6013" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6013</a> has been waiting for test resource for 3 days. I've to start manual test run to reproduce this issue.</p> <blockquote><p>it could be related to <a href="http://review.whamcloud.com/5820" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5820</a></p></blockquote> <p>Hi Hongchao, build <a href="http://build.whamcloud.com/job/lustre-master/1381/" class="external-link" target="_blank" rel="nofollow noopener">http://build.whamcloud.com/job/lustre-master/1381/</a> does not contain the above patch.</p> <p>While I trying to reproduce the issue in this ticket on Rosso cluster, I hit <a href="https://jira.whamcloud.com/browse/LU-3175" title="recovery-mds-scale test_failover_mds: unlink ./clients/client1/~dmtmp/PWRPNT/PPTC112.TMP failed (Read-only file system)" class="issue-link" data-issue-key="LU-3175"><del>LU-3175</del></a>.</p> <p>in the logs of <a href="http://review.whamcloud.com/#change,6013" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6013</a></p> <p>the stripe data of the file used by "dd" is missed after failover, then it set the layout type of the file to Empty,<br/> which causes this error (-9, Bad file descriptor)</p> <p>...<br/> 00020000:00000001:0.0:1366159710.913287:0:8913:0:(lov_object.c:359:lov_fini_raid0()) Process entered<br/> 00020000:00000010:0.0:1366159710.913288:0:8913:0:(lov_object.c:362:lov_fini_raid0()) kfreed 'r0-&gt;lo_sub': 56 at ffff88007d4e6940.<br/> 00020000:00000010:0.0:1366159710.913290:0:8913:0:(lov_ea.c:131:lsm_free_plain()) slab-freed 'lsm-&gt;lsm_oinfo<span class="error">&#91;i&#93;</span>': 112 at ffff8800373d1840.<br/> 00020000:00000010:0.0:1366159710.913292:0:8913:0:(lov_ea.c:131:lsm_free_plain()) slab-freed 'lsm-&gt;lsm_oinfo<span class="error">&#91;i&#93;</span>': 112 at ffff8800373d17c0.<br/> 00020000:00000010:0.0:1366159710.913293:0:8913:0:(lov_ea.c:131:lsm_free_plain()) slab-freed 'lsm-&gt;lsm_oinfo<span class="error">&#91;i&#93;</span>': 112 at ffff8800373d1740.<br/> 00020000:00000010:0.0:1366159710.913293:0:8913:0:(lov_ea.c:131:lsm_free_plain()) slab-freed 'lsm-&gt;lsm_oinfo<span class="error">&#91;i&#93;</span>': 112 at ffff8800373d16c0.<br/> 00020000:00000010:0.0:1366159710.913294:0:8913:0:(lov_ea.c:131:lsm_free_plain()) slab-freed 'lsm-&gt;lsm_oinfo<span class="error">&#91;i&#93;</span>': 112 at ffff8800373d1640.<br/> 00020000:00000010:0.0:1366159710.913294:0:8913:0:(lov_ea.c:131:lsm_free_plain()) slab-freed 'lsm-&gt;lsm_oinfo<span class="error">&#91;i&#93;</span>': 112 at ffff8800373d15c0.<br/> 00020000:00000010:0.0:1366159710.913295:0:8913:0:(lov_ea.c:131:lsm_free_plain()) slab-freed 'lsm-&gt;lsm_oinfo<span class="error">&#91;i&#93;</span>': 112 at ffff8800373d1540.<br/> 00020000:00000010:0.0:1366159710.913296:0:8913:0:(lov_ea.c:133:lsm_free_plain()) kfreed 'lsm': 128 at ffff88007b7cbc40.<br/> 00020000:00000001:0.0:1366159710.913297:0:8913:0:(lov_object.c:369:lov_fini_raid0()) Process leaving<br/> 00000020:00001000:0.0:1366159710.913297:0:8913:0:(cl_object.c:896:cl_env_put()) 1@ffff88007a373138<br/> 00000020:00000010:0.0:1366159710.913298:0:8913:0:(lu_object.c:421:lu_global_key_fini()) kfreed 'info': 512 at ffff8800175e4200.<br/> 00000020:00000010:0.0:1366159710.913299:0:8913:0:(cl_object.c:1090:cl0_key_fini()) kfreed 'info': 600 at ffff8800435e3000.<br/> 00000008:00000010:0.0:1366159710.913300:0:8913:0:(osc_dev.c:135:osc_key_fini()) slab-freed '(info)': 984 at ffff88002ad267f0.<br/> 00020000:00000010:0.0:1366159710.913301:0:8913:0:(lov_dev.c:159:lov_key_fini()) slab-freed '(info)': 288 at ffff88007db76be8.<br/> 00000080:00000010:0.0:1366159710.913302:0:8913:0:(lcommon_cl.c:146:ccc_key_fini()) slab-freed '(info)': 352 at ffff88007db048e0.<br/> 00000080:00000010:0.0:1366159710.913303:0:8913:0:(vvp_dev.c:96:vvp_key_fini()) slab-freed '(info)': 488 at ffff88002ad28420.<br/> 00000020:00000010:0.0:1366159710.913304:0:8913:0:(lu_object.c:1531:keys_fini()) kfreed 'ctx-&gt;lc_value': 320 at ffff8800175e4400.<br/> 00000008:00000010:0.0:1366159710.913305:0:8913:0:(osc_dev.c:159:osc_session_fini()) slab-freed '(info)': 424 at ffff88007db05590.<br/> 00020000:00000010:0.0:1366159710.913305:0:8913:0:(lov_dev.c:183:lov_session_key_fini()) slab-freed '(info)': 400 at ffff88007db74508.<br/> 00000080:00000010:0.0:1366159710.913306:0:8913:0:(lcommon_cl.c:164:ccc_session_key_fini()) slab-freed '(session)': 184 at ffff88007a36b1f8.<br/> 00000080:00000010:0.0:1366159710.913307:0:8913:0:(vvp_dev.c:114:vvp_session_key_fini()) slab-freed '(session)': 104 at ffff88007a36a7b0.<br/> 00000020:00000010:0.0:1366159710.913308:0:8913:0:(lu_object.c:1531:keys_fini()) kfreed 'ctx-&gt;lc_value': 320 at ffff880030d75800.<br/> 00000020:00000010:0.0:1366159710.913309:0:8913:0:(cl_object.c:787:cl_env_fini()) slab-freed '(cle)': 168 at ffff88007a373138.<br/> 00020000:00000001:0.0:1366159710.913310:0:8913:0:(lov_object.c:653:lov_layout_change()) Process leaving (rc=0 : 0 : 0)<br/> 00020000:00000001:0.0:1366159710.913311:0:8913:0:(lov_object.c:731:lov_conf_set()) Process leaving<br/> 00020000:00000001:0.0:1366159710.913312:0:8913:0:(lov_object.c:735:lov_conf_set()) Process leaving (rc=0 : 0 : 0)<br/> 00000020:00000001:0.0:1366159710.913313:0:8913:0:(cl_object.c:329:cl_conf_set()) Process leaving (rc=0 : 0 : 0)<br/> 00000020:00001000:0.0:1366159710.913314:0:8913:0:(cl_object.c:896:cl_env_put()) 1@ffff88007a373e58<br/> 00000020:00000010:0.0:1366159710.913315:0:8913:0:(lu_object.c:421:lu_global_key_fini()) kfreed 'info': 512 at ffff88001fc15c00.<br/> 00000020:00000010:0.0:1366159710.913316:0:8913:0:(cl_object.c:1090:cl0_key_fini()) kfreed 'info': 600 at ffff880049dc7c00.<br/> 00000008:00000010:0.0:1366159710.913317:0:8913:0:(osc_dev.c:135:osc_key_fini()) slab-freed '(info)': 984 at ffff88007b751418.<br/> 00020000:00000010:0.0:1366159710.913318:0:8913:0:(lov_dev.c:159:lov_key_fini()) slab-freed '(info)': 288 at ffff88007a370728.<br/> 00000080:00000010:0.0:1366159710.913319:0:8913:0:(lcommon_cl.c:146:ccc_key_fini()) slab-freed '(info)': 352 at ffff88007a36f5e0.<br/> 00000080:00000010:0.0:1366159710.913320:0:8913:0:(vvp_dev.c:96:vvp_key_fini()) slab-freed '(info)': 488 at ffff88007a36e238.<br/> 00000020:00000010:0.0:1366159710.913320:0:8913:0:(lu_object.c:1531:keys_fini()) kfreed 'ctx-&gt;lc_value': 320 at ffff88001765f400.<br/> 00000008:00000010:0.0:1366159710.913321:0:8913:0:(osc_dev.c:159:osc_session_fini()) slab-freed '(info)': 424 at ffff88002ac25e18.<br/> 00020000:00000010:0.0:1366159710.913322:0:8913:0:(lov_dev.c:183:lov_session_key_fini()) slab-freed '(info)': 400 at ffff88007db74828.<br/> 00000080:00000010:0.0:1366159710.913323:0:8913:0:(lcommon_cl.c:164:ccc_session_key_fini()) slab-freed '(session)': 184 at ffff88007a36b870.<br/> 00000080:00000010:0.0:1366159710.913323:0:8913:0:(vvp_dev.c:114:vvp_session_key_fini()) slab-freed '(session)': 104 at ffff88007a36a0c8.<br/> 00000020:00000010:0.0:1366159710.913324:0:8913:0:(lu_object.c:1531:keys_fini()) kfreed 'ctx-&gt;lc_value': 320 at ffff880017664000.<br/> 00000020:00000010:0.0:1366159710.913325:0:8913:0:(cl_object.c:787:cl_env_fini()) slab-freed '(cle)': 168 at ffff88007a373e58.<br/> 00000080:00000001:0.0:1366159710.913326:0:8913:0:(file.c:3236:ll_layout_conf()) Process leaving (rc=0 : 0 : 0)<br/> 00010000:00000001:0.0:1366159710.913327:0:8913:0:(ldlm_lock.c:211:ldlm_lock_put()) Process entered<br/> 00010000:00000001:0.0:1366159710.913328:0:8913:0:(ldlm_lock.c:244:ldlm_lock_put()) Process leaving<br/> 00010000:00000001:0.0:1366159710.913329:0:8913:0:(ldlm_lock.c:609:__ldlm_handle2lock()) Process entered<br/> 00000020:00000001:0.0:1366159710.913329:0:8913:0:(lustre_handles.c:172:class_handle2object()) Process entered<br/> 00000020:00000001:0.0:1366159710.913330:0:8913:0:(lustre_handles.c:195:class_handle2object()) Process leaving (rc=18446612134411306496 : -131939298245120 : ffff88007cff0600)<br/> 00010000:00000001:0.0:1366159710.913331:0:8913:0:(ldlm_lock.c:621:__ldlm_handle2lock()) Process leaving (rc=18446612134411306496 : -131939298245120 : ffff88007cff0600)<br/> 00010000:00000001:0.0:1366159710.913332:0:8913:0:(ldlm_lock.c:876:ldlm_lock_decref_internal()) Process entered<br/> 00010000:00010000:0.0:1366159710.913333:0:8913:0:(ldlm_lock.c:849:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff880037d22800 lock: ffff88007cff0600/0x6e4fd575de0e4a90 lrc: 3/1,0 mode: CR/CR res: 8589935616/3 bits 0x8 rrc: 1 type: IBT flags: 0x20000000000 nid: local remote: 0xb524751f9485880b expref: -99 pid: 8913 timeout: 0 lvb_type: 3<br/> 00010000:00000001:0.0:1366159710.913335:0:8913:0:(ldlm_lock.c:211:ldlm_lock_put()) Process entered<br/> 00010000:00000001:0.0:1366159710.913335:0:8913:0:(ldlm_lock.c:244:ldlm_lock_put()) Process leaving<br/> 00010000:00010000:0.0:1366159710.913337:0:8913:0:(ldlm_lock.c:931:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff880037d22800 lock: ffff88007cff0600/0x6e4fd575de0e4a90 lrc: 2/0,0 mode: CR/CR res: 8589935616/3 bits 0x8 rrc: 1 type: IBT flags: 0x20000000000 nid: local remote: 0xb524751f9485880b expref: -99 pid: 8913 timeout: 0 lvb_type: 3<br/> 00010000:00000001:0.0:1366159710.913341:0:8913:0:(ldlm_lock.c:312:ldlm_lock_add_to_lru()) Process entered<br/> 00010000:00000001:0.0:1366159710.913342:0:8913:0:(ldlm_lock.c:316:ldlm_lock_add_to_lru()) Process leaving<br/> 00010000:00000001:0.0:1366159710.913342:0:8913:0:(ldlm_lock.c:952:ldlm_lock_decref_internal()) Process leaving<br/> 00010000:00000001:0.0:1366159710.913343:0:8913:0:(ldlm_lock.c:211:ldlm_lock_put()) Process entered<br/> 00010000:00000001:0.0:1366159710.913343:0:8913:0:(ldlm_lock.c:244:ldlm_lock_put()) Process leaving<br/> 00000080:00000001:0.0:1366159710.913344:0:8913:0:(file.c:3339:ll_layout_lock_set()) Process leaving (rc=0 : 0 : 0)<br/> 00000080:00000001:0.0:1366159710.913345:0:8913:0:(file.c:3445:ll_layout_refresh()) Process leaving (rc=0 : 0 : 0)<br/> 00000080:00000001:0.0:1366159710.913346:0:8913:0:(vvp_io.c:1220:vvp_io_init()) Process leaving (rc=0 : 0 : 0)<br/> 00020000:00000001:0.0:1366159710.913347:0:8913:0:(lov_io.c:939:lov_io_init_empty()) Process entered<br/> 00020000:00000001:0.0:1366159710.913347:0:8913:0:(lov_io.c:968:lov_io_init_empty()) Process leaving (rc=1 : 1 : 1)<br/> 00000020:00000001:0.0:1366159710.913348:0:8913:0:(cl_io.c:178:cl_io_init0()) Process leaving (rc=1 : 1 : 1)<br/> 00000020:00000001:0.0:1366159710.913349:0:8913:0:(cl_io.c:239:cl_io_rw_init()) Process leaving (rc=1 : 1 : 1)<br/> 00000080:00000001:0.0:1366159710.913350:0:8913:0:(file.c:920:ll_file_io_generic()) Process leaving via out (rc=18446744073709551607 : -9 : 0xfffffffffffffff7)<br/> ...</p> <p>but the log of MDS is unavailable (it's at another node "wtm-15vm7"), and Yujian will help to collect it manually and will check it once the log is ready.</p> <p>Lustre Branch: master<br/> Lustre Build: <a href="http://build.whamcloud.com/job/lustre-master/1406/" class="external-link" target="_blank" rel="nofollow noopener">http://build.whamcloud.com/job/lustre-master/1406/</a><br/> Distro/Arch: RHEL6.3/x86_64<br/> Test Group: failover<br/> FAILURE_MODE=HARD</p> <p>Please look into this report: <a href="https://maloo.whamcloud.com/test_sets/3d4c2120-a849-11e2-ba78-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/3d4c2120-a849-11e2-ba78-52540035b04c</a></p> <p>this bug should be caused by the wrong size of "MD" in MDT,</p> <p>00000004:00000001:21.0:1366291257.871077:0:4881:0:(mdd_object.c:268:mdd_xattr_get()) Process entered<br/> 00000004:00000001:21.0:1366291257.871080:0:4881:0:(lod_object.c:373:lod_xattr_get()) Process entered<br/> 00000004:00000001:21.0:1366291257.871085:0:4881:0:(lod_object.c:377:lod_xattr_get()) Process leaving (rc=176 : 176 : b0)<br/> 00000004:00000001:21.0:1366291257.871087:0:4881:0:(mdd_object.c:281:mdd_xattr_get()) Process leaving (rc=176 : 176 : b0)<br/> 00000004:00020000:21.0:1366291257.871089:0:4881:0:(mdt_lvb.c:158:mdt_lvbo_fill()) lustre-MDT0000: expected 176 actual 128.<br/> 00000004:00000001:21.0:1366291257.880256:0:4881:0:(mdt_lvb.c:159:mdt_lvbo_fill()) Process leaving via out (rc=18446744073709551582 : -34 : 0xffffffffffffffde)</p> <p>1, the default value of mdt_device-&gt;mdt_max_mdsize is 128bytes,<br/> #define MAX_MD_SIZE (sizeof(struct lov_mds_md) + 4 * sizeof(struct lov_ost_data)</p> <p>2, before failover, the MD size is changed to 176bytes = (sizeof(struct lov_mds_md) + 6 * sizeof(struct lov_ost_data)<br/> mdt_device-&gt;mdt_max_mdsize is updated accordingly (see mdt_attr_get_lov, will update "mdt_max_mdsize" in "getattr" request)</p> <p>3, after failover, the new MDT doesn't know the actual mdt_max_mdsize, and still use the default value, then client calls ll_layout_refresh<br/> to get the MD and the MDT will failed with <del>ERANGE for there is still no "getattr" request to update mdt_device</del>&gt;mdt_max_mdsize!</p> <p>the patch is tracked at <a href="http://review.whamcloud.com/#change,6102" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6102</a></p> <p>A patch for master branch to gather the logs on passive server nodes in failure configuration: <a href="http://review.whamcloud.com/6112" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/6112</a>.</p> <p>Not closing this bug until the redundant getxattr call has been cleaned up per inspection comments.</p> <p>Efficiency of solution will be improved under <a href="https://jira.whamcloud.com/browse/LU-2818" title="Failure on test suite parallel-scale-nfsv4 test_compilebench: (mdt_lvb.c:126:mdt_lvbo_fill()) ASSERTION( rc == 0 ) failed" class="issue-link" data-issue-key="LU-2818"><del>LU-2818</del></a>. Current solution sufficient for 2.4</p> <p>I don't know how it got passed you build system but patch <a href="http://review.whamcloud.com/#change,6102" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6102</a> is missing the<br/> function fid_build_from_res_name.</p> <p>lustre-2.4.0/lustre/mdt/mdt_handler.c: In function ‘mdt_intent_layout’:<br/> lustre-2.4.0/lustre/mdt/mdt_handler.c:3754: error: implicit declaration of function ‘fid_build_from_res_name’</p> <p>I found it. Patch <a href="http://review.whamcloud.com/#change,4501" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,4501</a> removed the fid_build_from_res_name function. Creating patch and testing... </p> <p>Created a patch to fix this issue.</p> <p><a href="http://review.whamcloud.com/#change,6566" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6566</a></p> <p>James please could you open a new ticket to track this bug? It sounds like what you are describing is a regression introduced into this work (which was included in 2.4.0) by the work from <a href="https://jira.whamcloud.com/browse/LU-2193" title="lvbo_init failed for resource XXX: rc -2, after recovery" class="issue-link" data-issue-key="LU-2193"><del>LU-2193</del></a> (which happened since 2.4.0) so it would be easier to create a new ticket and link it to the other two related tickets. It can get really confusing trying to work out the situation for a given bug if there are commits to it spanning release boundaries.</p> Development Rank 1|hzvnjj: Rank (Obsolete) 7628 Severity