https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5433 Lustre <p>I noticed this manage appeared to be a bit incorrect and could use some editing in general.</p> <p>One thing that caught my eye as not quite right:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> HS_EXISTS A file copy exists in HSM backend. </pre> </div></div> <p>That flag only indicates the file has been assigned an HSM archive id, and does not mean it's been copied yet.</p> <p>Patch in progress. </p> LU-5433

Man page for llapi_hsm_state_get(3) needs some clarification

Bug Minor Resolved Fixed Robert Read Robert Read Wed, 30 Jul 2014 23:52:46 +0000 Tue, 22 Dec 2015 03:31:33 +0000 Mon, 22 Jun 2015 23:13:42 +0000 Lustre 2.8.0 0 5 <p><a href="http://review.whamcloud.com/#/c/11283/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/11283/</a></p> <p>&gt; That flag only indicates the file has been assigned an HSM archive id, and does not mean it's been copied yet.</p> <p>That's true but you're losing an important point which is: it does mean that a copy could have been started. But failed for some reasons. <br/> When this flag's set, administrators should consider a copy possibly exists in the backend pointed by archive_id. Especially if you are removing file from Lustre, you must take care of this flag. I'm afraid you lose this aspect if you change the documentation this way.</p> <p>I agree that flag indicates a file copy may or may not have been started, but there is no way to tell. If a copy exists in the backend and the ARCHIVE flag is not set, then it should be the backend's responsibility to clean this up, not the admin's. </p> <p>But the backend has no access to Lustre flags. Actually the backend is mostly Lustre agnostic. Those stale copies should be cleaned when the corresponding Lustre file is removed. This is the purpose of HS_EXISTS.</p> <p>It seems to me the copytool is the arbiter between the backend and Lustre state, and should be responsible for handling errors correctly. My interpretation of the flags is HS_ARCHIVED is set once a complete copy has been made in the backend, so it should only be possible for stale objects to exist of HS_ARCHIVED is also set. If only HS_EXISTS is set, then it seems one of several things could be true:</p> <p>1) Nothing has been done to this file because copytool hasn't received the action request yet.<br/> 2) A copy is currently in progress<br/> 3) A copytool received action request but failed, and copytool/backed should arrange for proper cleanup</p> <p>In case a problem occurs during the copy (backend outage, Lustre crash, copytool crash...) the copytool may not be able to clean the backend properly by itself.<br/> Making the backend responsible for cleaning may not be appropriate or not possible, as it can be a very expensive to scan the whole backend namespace to search for aborted copy files (also, it must have a way to distinguish a regular copy from an aborted copy...).</p> <p>Cleaning is not a problem if an entry is archived again: the copytool can do the cleaning of the previously aborted copy at this time. But, there is a leakage if the entry is removed from lustre in the meanwhile. In this case, HS_EXISTS flag is relayed in UNLINK changelog record so the Policy Engine is aware it must trigger a HSM_REMOVE action. This way, even a failed copy is correctly cleaned in the backend after file removal.</p> <p>Does this mean that whenever HS_EXISTS is not set, the archive_id can be safely ignored? And if so, should it be still displayed by "lfs hsm_state" after a "lfs hsm_remove"?</p> <p>&gt; Does this mean that whenever HS_EXISTS is not set, the archive_id can be safely ignored?</p> <p>Yes, but in theory this should not happened. archive_id is set at the same time than HS_EXISTS.</p> <p>&gt; And if so, should it be still displayed by "lfs hsm_state" after a "lfs hsm_remove"?</p> <p>A successful hsm_remove should clear both of them.<br/> This is strange if you ran this command and archive_id was kept.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/11283/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11283/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5433" title="Man page for llapi_hsm_state_get(3) needs some clarification" class="issue-link" data-issue-key="LU-5433"><del>LU-5433</del></a> doc: update llapi_hsm_state_get manpage<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 0572733bf6e41b35331358c33289ab46e0182878</p> <p>Landed on master.</p> Development Rank 1|hzwsp3: Rank (Obsolete) 15134 Severity