https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3833 Lustre <p>The copytool does not check for sparseness of files. Hence truncating a file to a very large size will create a potential DoS if that file is archived.</p> LU-3833

HSM POSIX copytool sparse file handling

Improvement Major Open Unresolved WC Triage John Hammond HSM Fri, 23 Aug 2013 20:51:29 +0000 Wed, 26 Jan 2022 19:40:42 +0000 Lustre 2.5.0 0 7 <p>This could probably be avoided to some extent by having the policy engine make decisions based on the number of allocated blocks rather than using the size of the file. It may also make sense for Robin Hood to allow specifying in the policy to avoid (or reduce) the priority for handling sparse files (possibly as a factor of the "sparseness") if the archive does not handle sparse files well. Otherwise, some file formats like HDF5 may have very large sparse address spaces that would fill both the archive and Lustre if handled poorly.</p> <p>A second-level enhancement would be to use FIEMAP to copy the file sparsely to/from the archive (assuming the archive itself has efficient sparse file handling).</p> <p>John, Andreas, sorry for the late update on this but since this looks more as an improvement/feature ...<br/> So back to your proposals and ideas, I will try to work in both directions :</p> <p> _ optimize (to be checked, with help from T.Leibovici, if still to be done?) RobinHood handling of sparse files, with associated Policy configuration parameters, to be based on blocks usage vs file-size.</p> <p> _ enhance copytool to use FIEMAP if archive filesystem support it. BTW, I am not sure if the actual HSM-Restore feature already use FIEMAP based on original+released file layout?</p> <p>I will try to come up with some basic changes and may be more detailed thoughts soon now.</p> <p>Mike Pershin (mpershin@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/40530" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40530</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-3833" title="HSM POSIX copytool sparse file handling" class="issue-link" data-issue-key="LU-3833">LU-3833</a> hsm: lhsmtool to handle sparse files<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: ae180a1080dc1cb3990d8f53caee95e11a160248</p> Related LU-13397 LU-6848 Development Rank 1|hzvz6n: Rank (Obsolete) 9911 Severity