https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9540 Lustre <p>The time required to submit new HSM requests grows linearly with the size of the HSM actions log. This is because we first run <tt>hsm_find_compatible()</tt> on the HAL to see if there are already existing actions running or waiting on the files in the HAL. We should avoid this by keeping an in-memory such cache of actions indexed by FID.</p> LU-9540

cache HSM actions in memory by FID

Bug Minor Open Unresolved WC Triage John Hammond Fri, 19 May 2017 14:56:32 +0000 Mon, 8 Jun 2020 22:54:24 +0000 0 8 <p>This seems like vast overkill to a dubious function - <br/> hsm_find_compatible can simply be skipped for everything except CANCEL.<br/> Nikitas is working on a patch.</p> <p>&gt; hsm_find_compatible can simply be skipped for everything except CANCEL.</p> <p><tt>hsm_find_compatible()</tt> is used to deduplicate restore requests (from multiple clients, say) for a given file.</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=jhammond" class="user-hover" rel="jhammond">jhammond</a>, another developer within HPE created a patch to skip calling hsm_find_compatible_cb() when all requests in a hal are archives or restores; we are trying to determine if this is safe to do. The CDT will set HS_EXISTS before sending an archive request to a copytool in mdt_hsm_agent_send ()&#45;&gt;mdt_hsm_add_hal(), so an archive request for the same file but for a different archive backend will not be sent as mdt_hsm_agent_send()&#45;&gt;mdt_hsm_is_action_compat() will return false; the duplicate request will be failed in the llog and should be removed by the CDT thread later due to timeout. I think by changing the checks in the latter function to also return false if the archive ids are the same unless HS_DIRTY is set should serialize archive requests in all cases? Restore requests seem to take the layout lock on the file in mdt_hsm_register_hal()-&gt;cdt_restore_handle_add() before being added to the llog so I think this should serialize them as well, although it blocks the caller, e.g. lfs, so I am not sure if it's ideal.</p> <p>It already seems possible to have duplicate archive requests added to the llog or duplicate restores racing for the layout lock without applying the aforementioned patch as multiple RPCs can race in mdt_hsm_register_hal(), so I think the patch wouldn't be introducing a new race but making the existing one more likely. Implementing the cache in this ticket sounds like a more elegant solution, but do you think skipping the call to hsm_find_compatible_cb() in the way described would be safe? We can submit the patch to Gerrit if needed, of course.</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=jhammond" class="user-hover" rel="jhammond">jhammond</a>, I have submitted the aforementioned patch at <a href="https://review.whamcloud.com/#/c/38867/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/38867/</a>.</p> Related LU-13651 Development Rank 1|hzzd9z: Rank (Obsolete) 9223372036854775807 Severity