https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5757 Lustre <p>Creating, opening and executing the following commands on a regular file:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>rc = llapi_hsm_state_set_fd(fd, HS_EXISTS, 0, -1789); rc = llapi_hsm_state_set_fd(fd, HS_DIRTY, 0, 0); rc = llapi_hsm_state_set_fd(fd, 0, HS_DIRTY, 0); rc = llapi_hsm_state_set_fd(fd, HS_ARCHIVED, 0, 0); rc = llapi_hsm_state_set_fd(fd, HS_RELEASED, 0, 0); rc = llapi_hsm_state_set_fd(fd, HS_NORELEASE, 0, 0); rc = llapi_hsm_state_set_fd(fd, HS_NOARCHIVE, 0, 0); rc = llapi_hsm_state_set_fd(fd, HS_LOST, 0, 0); rc = llapi_hsm_state_set_fd(fd, 0x00080000, 0, 0); rc = llapi_hsm_state_set_fd(fd, 0x80000000, 0, 0); </pre> </div></div> <p>results in this:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># ../utils/lfs hsm_state /mnt/lustre/hsm_check_test /mnt/lustre/hsm_check_test: (0x8008007d) released exists archived never_release never_archive lost_from_hsm, archive_id:-1789 </pre> </div></div> <p>Some of these flags should not be user settable. Unknown flags should not be set. The archive_id should not be anything but &gt;=0 (or 1?) and &lt;=32.</p> LU-5757

hsm: userspace can set about any HSM flags. Checks are inexistant.

Bug Minor Resolved Fixed Bruno Faccini Frank Zago hsm Thu, 16 Oct 2014 17:08:09 +0000 Thu, 9 Mar 2017 20:33:10 +0000 Wed, 25 Mar 2015 17:09:52 +0000 Lustre 2.7.0 Lustre 2.5.3 Lustre 2.8.0 0 10 <p>Just to confirm - this can only be done on files a user owns, or by root? It definitely makes sense to add better checks for the HSM input (see also <a href="https://jira.whamcloud.com/browse/LU-2714" title="HSM: add sanity checks for incoming RPCs" class="issue-link" data-issue-key="LU-2714"><del>LU-2714</del></a>), so a general review of all such input would be welcome.</p> <p>This test was done as root.</p> <p>I checked with a user, and most of these commands are reject with -EPERM. I think that's wrong too because -EPERM should only be used when the user doesn't have the correct permissions on that file.</p> <p>Why is archive_id limited to max 32?</p> <p>I believe the LOST flag is intended to be set by userspace, since this reflects backend state that the kernel is unaware of. It is also useful to be able to clear the flags from user space to allow a file to be moved to a different HSM archive backend. (Although it would be better to clear flags automatically when the archive_id is changed, though that would be a behavior change and might break something.)</p> <p>Robert,</p> <p>I think the 32 archive ids is a limitation of Lustre. lhsmtool_posix.c has this:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> 60 /* copytool uses a 32b bitmask field to register with kuc 61 * archive num = 0 =&gt; all 62 * archive num from 1 to 32 63 */ 64 #define MAX_ARCHIVE_CNT (sizeof(__u32) * 8) </pre> </div></div> <p>Interesting. This appears to only be a limitation of the kuc interface. If a copytool doesn't specify an archive id when it calls register (i.e. passes a 0), then it seems it can handle any archive id. </p> <p>I confirm this limit to 32 comes from kuc protocol/structures. Seeing the use cases, 32 looks ok and mainly all the site will have one backend archive system and so will always use the default (0 = all)</p> <p>Test for this case (test51): <a href="http://review.whamcloud.com/12836/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/12836/</a></p> <p>Faccini Bruno (bruno.faccini@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/13337" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13337</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5757" title="hsm: userspace can set about any HSM flags. Checks are inexistant." class="issue-link" data-issue-key="LU-5757"><del>LU-5757</del></a> hsm: strengthen checks for flags and archive id<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 9379ce022fe908f3168987bfbc12214426f57fe8</p> <p>Just pushed master pach <a href="http://review.whamcloud.com/13337" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13337</a> which should allow for HSM flags and archive-id values to be checked.</p> <p>Thanks Bruno. I've updated test test in <a href="https://jira.whamcloud.com/browse/LU-5757" title="hsm: userspace can set about any HSM flags. Checks are inexistant." class="issue-link" data-issue-key="LU-5757"><del>LU-5757</del></a> (not pushed yet) and it's better.</p> <p>But should root even be allowed to set/unset the following flags: HS_DIRTY, HS_ARCHIVED and HS_RELEASED?</p> <p>Patch to add since <a href="https://jira.whamcloud.com/browse/LU-5732" title="hsm: sanity check for sane progress extent value" class="issue-link" data-issue-key="LU-5732"><del>LU-5732</del></a> went upstream</p> <p>Thanks to warn me (and to provide your add-on proposal) about the fact that some of your new tests in your patch for <a href="https://jira.whamcloud.com/browse/LU-5732" title="hsm: sanity check for sane progress extent value" class="issue-link" data-issue-key="LU-5732"><del>LU-5732</del></a> will fail. I don't know why but I was wrongly assuming that you had already anticipated the final/expected behavior and error returns from this patch in your new tests ... BTW, since I was planning to either run your new tests against this patch, or apply/relate mine on top of your or now that yours has been integrated to rebase mine, hopefully auto-tests would have also warned me!</p> <p>Concerning your remark about root being or not allowed to set/unset HS_DIRTY, HS_ARCHIVED and HS_RELEASED, my feeling is that yes it should be allowed, mainly to help fixing unusual/inconsistent situations. But this can be discussed!</p> <p>I agree there are use cases for manually setting HS_DIRTY and HS_ARCHIVED, but just setting the HS_RELEASED flag is not enough to really make a file "released". The layout also needs to changed, and I don't see a way to do that directly. However, if one can clear HS_DIRTY and manually set HS_ARCHIVED, then in theory the HUA_RELEASE request should properly mark the file as released. If the file is somehow corrupted then it's also possible to use llapi_hsm_import() to recreate the released file. </p> <p>Robert, thanks to detail this. And Yes, using the import feature along with manual flags setting was some of the repair actions I thought about to be helpful in some cases.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/13337/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13337/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5757" title="hsm: userspace can set about any HSM flags. Checks are inexistant." class="issue-link" data-issue-key="LU-5757"><del>LU-5757</del></a> hsm: strengthen checks for flags and archive id<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 32bd5051a518c57e35f51b7f3c7f739b5ef91b25</p> <p>Landed for 2.8</p> Related LU-6453 Development Rank 1|hzwyqn: Rank (Obsolete) 16155 Severity