https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13772 Lustre <p>We recently had the case of an MDT whose changelog records were not being processed and cleared as they should have been. We quickly reached a point where the whole catalog was full, and we had little choice but to deregister the changelog reader to resume production.</p> <p>We used <tt>lctl --device lustre-MDT0000 changelog_deregister cl1</tt> for that, and it took 3 days to complete. Considering we only had a single changelog reader registered, and our goal was to simply garbage collect every changelog record, it feels wasteful that we should wait 3 days for something that essentially deletes a few files on the MDT.</p> <p>Would it be possible to speed up this process?</p> <p>It would be nice that this works by special-casing <tt>lctl changelog_deregister</tt> when there is only one reader registered, but I think a new command (eg. <tt>lctl changelog_delete_everything</tt>, <tt>lctl changelog_reset</tt>, ...) would also be satisfying.</p> LU-13772

mdt: changelog_deregister takes too long

Improvement Minor Resolved Fixed Mikhail Pershin Quentin Bouget Thu, 9 Jul 2020 09:42:59 +0000 Thu, 9 Sep 2021 18:45:57 +0000 Thu, 9 Sep 2021 18:45:57 +0000 Upstream 0 4 <p>This is important to us, because we use robinhood, and during those 3 days, robinhood essentially works in the dark.</p> <p>We could launch periodic scans on our filesystem, but scans take a while, and they can still miss things.</p> <p>Quentin</p> <p>Is this something that you plan to work on or just a suggestion for someone else to work on?</p> <p>Peter</p> <p>Hi Peter,</p> <p>Just a suggestion. Although it may become a request one day.<br/> We will sort this out with our support team when/if the time comes.</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=tappro" class="user-hover" rel="tappro">tappro</a> could you look at this after <a href="https://jira.whamcloud.com/browse/LU-13055" title="add ability for named Changelog consumers" class="issue-link" data-issue-key="LU-13055"><del>LU-13055</del></a> and in <a href="https://jira.whamcloud.com/browse/LU-13338" title="add changelog mask per registered user" class="issue-link" data-issue-key="LU-13338"><del>LU-13338</del></a>?</p> <p>It makes sense to have a fast-path for deleting changelog records. If the last index in the changelog is less than the lowest user, then all of the records should be deleted, and the whole file can be removed immediately. That would speed up changelog removal by 50000x or so (a few writes to delete the file, instead of 64000 writes to cancel every record.</p> <p>It looks like this will coincidentally be fixed by patch <a href="https://review.whamcloud.com/43719" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43719</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-14688" title="Changelog cancel improvement" class="issue-link" data-issue-key="LU-14688"><del>LU-14688</del></a> mdt: changelog purge deletes plain llog</tt>" that was just pushed a few days ago.</p> <p>The patch <a href="https://review.whamcloud.com/43719" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43719</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-14688" title="Changelog cancel improvement" class="issue-link" data-issue-key="LU-14688"><del>LU-14688</del></a> mdt: changelog purge deletes plain llog</tt>" has been landed for 2.14.52.</p> Duplicate Related LU-14699 LU-14688 Development Epic/Theme changelog Rank 1|i014sn: Rank (Obsolete) 9223372036854775807