https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3362 Lustre <p>Need disaster recovery support for HSM in 2.5</p> LU-3362

HSM - Disaster Recovery Support - Master Landings

New Feature Critical Resolved Fixed Alex Zhuravlev Jodi Levi HSM Mon, 20 May 2013 20:29:15 +0000 Thu, 5 Sep 2013 18:53:04 +0000 Thu, 5 Sep 2013 18:53:04 +0000 Lustre 2.5.0 Lustre 2.5.0 0 10 <p>Meeting 2013-06-05<br/> Use Cases:<br/> 1. Need MDT backup and restore with what is in archive (import by FID)<br/> 2. Restore into same file system - import file that was deleted (import by FID)<br/> 3. Brand new file system formatted - have archive available and import into new file system (import by FID)<br/> 4. Import into a file system which already has data (requires FID rebinding)<br/> Focus for 2.5 will be on the first 3 Use Cases<br/> Technical details will be discussed in meeting 2013-06-06 and added in this ticket<br/> Changes to Lustre will be done by Intel<br/> All other changes will be done by CEA</p> <p>This can be achieved by implementing a "rebind" operation in copytool (CEA).<br/> To avoid fid collisions when reimporting files to a new filesystem, we need a hack in FLDB to avoid re-using sequences of old fids (Intel).</p> <p>Implementation details:<br/> When recovering a filesystem from the archive, we must rebind archived entries to the newly created fids in Lustre.<br/> We must avoid collisions between old fids and new fids during this operation.</p> <p>Thus, to achieve distaster recovery, the following features are needed:<br/> 1) add the copytool a "rebind" feature. This is implemented as part of change 4737 <a href="http://review.whamcloud.com/#/c/4737/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/4737/</a> ("rebind" copytool option).<br/> 2) be able to get the max fid sequence referenced in the archive. This is also implemented as part of change 4737 <a href="http://review.whamcloud.com/#/c/4737/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/4737/</a> ("max_sequence" copytool option).<br/> 3) we must ensure that the newly formatted Lustre filesystem will not allocate fids in the old ranges.<br/> To achieve this, we pass this max_sequence to mkfs, so that all the ranges up to max_sequence are reserved in the FLDB.</p> <p>may I ask why FIDs need to be preserved?</p> <p>I have also been lobbying for not storing FIDs in the archive. Actually, CEA and I came up with a solution doing exactly this and not require preserving or remapping FIDs.</p> <p>That said, CEA thinks that this solution cannot be implemented by the feature freeze, so they want to go on with their current scheme which stores FID in the "archive". As a consequence, we either have to preserve original FIDs or re-map FIDs on the archive. Given that the archive is the slowest component, preserving FID sounds like the most reasonable approach.</p> <p>I see... the issue is that it's not mkfs who creates FLDB.. will try to figure out a solution.</p> <p>this is not supposed to be used with DNE?</p> <p>just to clarify .. if we don't need to support DNE, then we just insert (somehow) an &lt;reserved sequences&gt; -&gt; MDT#0 mapping into newly created FLDB.<br/> otherwise it'll be more difficult.</p> <p>Yes, you can assign the reserved sequence range to MDT0. It is enough to avoid collision.</p> <p><a href="http://review.whamcloud.com/#/c/7027/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/7027/</a></p> <p>Patch landed to master. Additional patch moved to a separate ticket for 2.6</p> Related LU-3608 LU-3565 Development Rank 1|hzvrdb: Rank (Obsolete) 8325