https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10384 Lustre <p>There is MDT-MDT connection problem after failover.</p> <p>Here is config for MDT0000 (no replace_nids applied for it)</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">#10 (224)marker 17 (flags=0x01, v2.5.1.0) lustre-MDT0000 ‘add osp’ Wed Aug 30 10:34:36 2017- #11 (088)add_uuid nid=192.168.0.112@tcp(0x20000c0a80070) 0: 1:192.168.0.112@tcp #12 (144)attach 0:lustre-MDT0000-osp-MDT0001 1:osp 2:lustre-MDT0001-mdtlov_UUID #13 (152)setup 0:lustre-MDT0000-osp-MDT0001 1:lustre-MDT0000_UUID 2:192.168.0.112@tcp #14 (088)add_uuid nid=192.168.0.113@tcp(0x20000c0a80071) 0: 1:192.168.0.113@tcp #15 (120)add_conn 0:lustre-MDT0000-osp-MDT0001 1:192.168.0.113@tcp #16 (136)modify_mdc_tgts add 0:lustre-MDT0001-mdtlov 1:lustre-MDT0000_UUID 2:0 3:1 #17 (224)END marker 17 (flags </pre> </div></div> <p>And MDT0001 config after replace_nids.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">#19 (224)marker 20 (flags=0x01, v2.5.1.0) lustre-MDT0001 ‘add osp’ Wed Aug 30 10:34:36 2017- #20 (088)add_uuid nid=192.168.0.113@tcp(0x20000c0a80071) 0: 1:192.168.0.113@tcp #21 (144)attach 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID #22 (152)setup 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.0.113@tcp #23 (136)modify_mdc_tgts add 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 #24 (224)END marker 20 (flags=0x02, v2.5.1.0) lustre-MDT0001 ‘add osp’ Wed Aug 30 10:34:36 2017- </pre> </div></div> <p>Replace nids doesn't add failover nid and add_conn string to config. This is the reason ops connection can not be established after failover.</p> <p>The solution is add option to replace_nids that adds failover record.</p> LU-10384

Replace nids doesn't add failover nid and add_conn string to config

Bug Critical Resolved Fixed Artem Blagodarenko Artem Blagodarenko patch Thu, 14 Dec 2017 16:27:44 +0000 Fri, 19 Apr 2019 15:16:18 +0000 Fri, 4 Jan 2019 05:02:28 +0000 Lustre 2.13.0 Lustre 2.12.1 0 7 <p>Artem Blagodarenko (c17828@cray.com) uploaded a new patch: <a href="https://review.whamcloud.com/30624" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30624</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10384" title="Replace nids doesn&#39;t add failover nid and add_conn string to config" class="issue-link" data-issue-key="LU-10384"><del>LU-10384</del></a> mgs: replace_nids large string and failover support<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: e5ac0db60e4d7772877f35b7a49b079f7031ccd3</p> <p>Hello <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=James_Nunez" class="user-hover" rel="James_Nunez">James_Nunez</a> <br/> I see such message in logs</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[16136.542599] LustreError: 10477:0:(mgc_request.c:1576:mgc_apply_recover_logs()) mgc: cannot find uuid by nid 10.2.8.156@tcp [16136.543707] Lustre: 10477:0:(mgc_request.c:1802:mgc_process_recover_nodemap_log()) MGC10.2.8.156@tcp: error processing recovery log lustre-mdtir: rc = -2 [16136.545012] LustreError: 10477:0:(mgc_request.c:2132:mgc_process_log()) MGC10.2.8.156@tcp: recover log lustre-mdtir failed, not fatal: rc = -2 </pre> </div></div> <p>10.2.8.156 - is new address that was applied by lctl replace_nids</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[16118.896695] Lustre: DEBUG MARKER: lctl replace_nids lustre-MDT0000 10.2.8.156@tcp [16119.429006] Lustre: DEBUG MARKER: lctl replace_nids lustre-MDT0001 10.2.8.156@tcp [16119.748693] Lustre: DEBUG MARKER: lctl replace_nids lustre-OST0000 10.2.8.156@tcp [16120.067065] Lustre: DEBUG MARKER: lctl replace_nids lustre-OST0001 10.2.8.156@tcp </pre> </div></div> <p>I can not be sure now if my patch have no influence to 108a and 108b tests falls. I am going to investigate the tests hangs in this issue. Because I have no ready zfs-based installation here, and testing system can easily reproduce this issue, can I ask to support me sharing some extra finales? <br/> First, I need files from CONFIGS directors on MGS to check if replace_nids precessed well.</p> <p>Thanks,</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=jamesanunez" class="user-hover" rel="jamesanunez">jamesanunez</a> Maloo set -1 to my patch. I checked locally config_sanity test_32d is failed with "rmmod: ERROR: Module zfs is in use" with and without my patch. Test test_75 is always passed in my local box (with/without my patch).</p> <p>Can you verify patch? Thanks. </p> <p>Same result for test_32a. Failed with/without my patch.</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/30624/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30624/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10384" title="Replace nids doesn&#39;t add failover nid and add_conn string to config" class="issue-link" data-issue-key="LU-10384"><del>LU-10384</del></a> mgs: replace_nids large string and failover support<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 09da1564d3794ca7b82e1c1791da253bee6178d4</p> <p>Landed for 2.13</p> <p>Minh Diep (mdiep@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/34296" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34296</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10384" title="Replace nids doesn&#39;t add failover nid and add_conn string to config" class="issue-link" data-issue-key="LU-10384"><del>LU-10384</del></a> mgs: replace_nids large string and failover support<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 7bc57e9b3bf9f4d77126133d68e40c8031934385</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/34296/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34296/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10384" title="Replace nids doesn&#39;t add failover nid and add_conn string to config" class="issue-link" data-issue-key="LU-10384"><del>LU-10384</del></a> mgs: replace_nids large string and failover support<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 97654820f8a60b1752b92b79cd6bc254a4e48958</p> Duplicate Related LUDOC-523 Development Rank 1|hzzpe7: Rank (Obsolete) 9223372036854775807 Severity