https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4966 Lustre <p>If some server registered successfully on MGS, but it got an error registration reply (MGS revoking config locks timeout or other networking problems), then the server will always get -EADDRINUSE error when it try to register next time, because the server index has been occupied on MGS in the first registration.</p> <p>Current solution for above situation is to use writeconf option to force registration.</p> <p>We need to get this improved and make MGS able to handle this gracefully.</p> LU-4966

handle server registration errors gracefully

Improvement Minor Open Unresolved WC Triage Niu Yawei llnl Mon, 28 Apr 2014 08:11:58 +0000 Tue, 5 Dec 2023 00:37:29 +0000 0 8 <p>I think if MGS save server UUID along with the server index, the it can tell if the registration (acquire for an occupied index) come from same server.<br/> And looks MGS now keeps server index bitmap in memory only, it needs be saved in disk as well.</p> <p>Andreas/Alex, any suggestions? Thanks.</p> <p>this issue was mentioned by Chris in <a href="https://jira.whamcloud.com/browse/LU-1257" title="OST registration snafu" class="issue-link" data-issue-key="LU-1257"><del>LU-1257</del></a> and at the moment it's not clear whether this specific issue is a major one for LLNL. it'll be hard to get this fixed in 2.6 due to amount of changes ?</p> <p>Alex, I think Niu was asking for ideas on how this might best be fixed.</p> <p>Niu's idea makes sense to me. I spent some time experimenting with it.</p> <p>A real UUID (e.g., "a53bc5ba-687b-4091-fb0b-61489785f247") could easily be generated by back-end-independent mkfs.lustre code and stored in a back-end-specific way (e.g., in ldiskfs "mountdata" or as a ZFS dataset property). (ZFS has pool IDs, but those are pool properties and are only 64-bit.)</p> <p>Current master code always passes empty strings in mti_uuid. It would be nice if real UUIDs could be packed into that field. However, experiments showed:</p> <ul> <li>MDT OSPs for OSTs would send MDS_CONNECTs to OSTs, because client_obd_setup() depends on "OST" in UUIDs to determine whether an OSP is for an OST or an MDT.</li> <li>Clients would not be able to connect to MDTs, because mgs would put fake UUIDs (e.g., "lustre-MDT0000_UUID") into MDT logs but real UUIDs into the client log.</li> </ul> <p>A possible solution is:</p> <ul> <li>Start generating real UUIDs for any newly formatted targets.</li> <li>Send real UUIDs via mti_uuid.</li> <li>MGT checks and stores real UUIDs somewhere, but keeps putting fake UUIDs into logs.</li> <li><b>Newly formatted targets must talk with new mgs code.</b></li> </ul> <p>There is space in the last_rcvd file to store the UUID, but that has the potential problem that this file may be deleted if there are problems with recovery. </p> <p>As for the OST detection in the connection cide, it would be possible to store the target type in the last byte of the UUID or similar (e.g. the ASCII "O" or "M") and still make the rest of the UUID random. </p> <p>Was this issue somehow adressed in latest versions ?<br/> We are still seeing this problem when installing configurations with large amount of OSTs (or when modifiying it after a writeconf) and have to take care manually of it by starting OSTs one after another.</p> Related LU-1257 LU-9838 LU-15112 LU-14928 LU-17240 Development Rank 1|hzwl47: Rank (Obsolete) 13736