https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4140 Lustre <p>When volatile files are created, a CREATE changelog record is emitted. But no record says that file was also immediately unlinked.</p> <p>Either we need a UNLINK record for that FID, or a special flag in CLOSE record.</p> <p>By the way, I do not think lustre_rsync is handling volatile files, but more especially it should handled LAYOUT records rather than volatile files.</p> LU-4140

Volatile files have CREATE changelog record but nothing to tell it is removed.

Bug Critical Resolved Fixed Bruno Faccini Aurelien Degremont HSM Thu, 24 Oct 2013 13:56:19 +0000 Thu, 19 Feb 2015 23:25:12 +0000 Thu, 5 Jun 2014 15:34:41 +0000 Lustre 2.4.2 Lustre 2.5.1 Lustre 2.6.0 Lustre 2.5.4 0 12 <p>Bruno,<br/> Could you have a look and comment on this one?<br/> Thank you!</p> <p>I am of the opinion that the CREATE record should not be added for volatile files at all because those files are not supposed to have any names at all, they are by definition invisible in the namespace (or should be).</p> <p>As such we should just fix the bug of adding the create record</p> <p>Indeed, let's not add CREATE changelog for volatile files.</p> <p>Ok! Will do.</p> <p>Please note that all other Changelog record are also raised when dealing with a volatile file. If you do SETATTR or change HSM flags, etc... this will raise also records for the volatile FID. If we just remove CREATE record, there will be other records for a FID that has never been created, from a CL point of view</p> <p>Sorry I am late on this, but if I fully understand, we just want to avoid any ChangeLog record for volatile files ??…</p> <p>After a volatile is created, many action like layout swap, or access by FID will involve the volatile FID. So removing all the CL will be difficult. IMO we have to keep the CREATE event, with a Volatile flag or followed by an unlink event. The issue with the second is that the unlink is fake, in particular the directory mtime will not change. So an unlink event with a volatile flag ?</p> <p>Ok, I am VERY late on this ticket, and by re-reading it, I understand that even if generated, the CL records for the Volatile are useless, but am I right ?</p> <p>If yes, I wonder if it could be possible to easily detect them from the CL layer/routines and simply avoid them all ?</p> <p>If not, I will be back with the "volatile flag" proposal, and this for all CL records/events ?</p> <p>&gt; even if generated, the CL records for the Volatile are useless, but am I right ?<br/> yes today but may be in the future</p> <p>&gt;If yes, I wonder if it could be possible to easily detect them from the CL layer/routines and simply avoid them all ?<br/> no because a volatile file is a std unlinked file</p> <p>&gt;If not, I will be back with the "volatile flag" proposal, and this for all CL records/events ?<br/> I think this is the right solution</p> <blockquote> <p>even if generated, the CL records for the Volatile are useless, but am I right ?</p></blockquote> <p>It depends what you want to track. So far, there is no important feature requiring them. But you can imagine several use cases where dropping them could be an issue. Especially on different accounting cases. This offers a way to create stealth files and uses disk space without any changelog speaking of it.</p> <blockquote> <p>no because a volatile file is a std unlinked file</p></blockquote> <p>It could be detected as it has a special name, but this would be ugly.</p> <blockquote> <p>If not, I will be back with the "volatile flag" proposal, and this for all CL records/events ?</p></blockquote> <p>Currently the issue is more than some events are raised but not others. In this specific cases no REMOVE events for VOLATILE file. It seems the issue is no CL events for open-unlink files, even it they are not volatile ones. To be checked but maybe this is the real bug.</p> <p>This is something important IMHO, but it does not need to be a blocker for 2.5.1</p> <p>Hello Aurelien, I am back on this now, and I will check the issue you raised about "no CL events for open-unlink files, even it they are not volatile ones", and let you know soon.</p> <p>I have been able to confirm that this issue is linked to the fact that volatile file usage is causing MDT objects orphans/leak. I will (finaly/hopefully?) have time now to push a patch to fix ...</p> <p>I've reopened <a href="https://jira.whamcloud.com/browse/LU-3696" title="sanity test_17m, test_17n: e2fsck unattached inodes failure" class="issue-link" data-issue-key="LU-3696"><del>LU-3696</del></a> to track the fix for MDT object leak for volatile files separately. That problem is critical since it is leaking space in the filesystem (both on the MDT and OST) for every volatile file used and needs to be back-ported to 2.4.x and 2.5.x. The presence/absence of a changelog record for volatile files is a much more complex issue and I think it can be fixed and landed separately.</p> <p>I pushed <a href="http://review.whamcloud.com/10179" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10179</a> to try and fix the volatile file problem, and also avoid creating ChangeLog entries for volatile files. </p> <p>I did some extended testing and it appears that with this patch there is no longer a "i_am_nobody" orphan inode left for each hsm_release operation nor a ".^L^S^T^R:VOLATILE:: ..." one for each hsm_restore operation, this is for <a href="https://jira.whamcloud.com/browse/LU-3696" title="sanity test_17m, test_17n: e2fsck unattached inodes failure" class="issue-link" data-issue-key="LU-3696"><del>LU-3696</del></a>.. <br/> And I also checked it also fixes/avoids the unecessary reporting of ChangeLog events for these volatile files, this is for <a href="https://jira.whamcloud.com/browse/LU-4140" title="Volatile files have CREATE changelog record but nothing to tell it is removed." class="issue-link" data-issue-key="LU-4140"><del>LU-4140</del></a>.</p> <p>Patch landed to Master.</p> Related LU-3696 Development Rank 1|hzw6rb: Rank (Obsolete) 11237 Severity