https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-7749 Lustre <p>during migration we put old objects onto /PENDING and leave it there till the next reboot (PENDING cleanup). this approach has a number of problems:<br/> 1) space (inodes and directories) aren't released<br/> so if one wanted to rebalance/free space on MDT will have to reboot it<br/> 2) PENDING/ may become huge slowing down regular unlinks<br/> this is especially bad as it never shrink</p> LU-7749

DNE3: migrated orphan survive till next reboot

Bug Minor Open Unresolved WC Triage Alex Zhuravlev dne3 Fri, 5 Feb 2016 11:54:19 +0000 Tue, 7 Apr 2020 22:29:05 +0000 Lustre 2.8.0 0 6 <p>Alex, what do you propose for a solution here? There is definitely a need to keep migrated inodes on the current MDT for some time, to avoid access races as well as things like NFS file handles (which contain the FID). The problem with NFS file handles is that they can be long lived.</p> <p>I agree that keeping the old inode for a long time isn't necessarily desirable, so possibly an LRU and/or aging to remove them? Should this be seconds, minutes, hours?</p> <p>You had previously also mentioned that the inodes are deleted from PENDING after a reboot, which I guess is true since nothing holds them open after recovery. It would be nice to fix this as well, though I don't think it is a blocker.</p> <p>In theory, we don't need to actually keep this redirection on the disk. It could be only in memory (at least currently, since PENDING isn't really persistent across recovery), but then it would pin a lot of RAM, which may be even less desirable. It would also be possible to have a redirection index with <span class="error">&#91;key=old_fid,value=new_fid&#93;</span> mappings, which would take less space than keeping a whole inode around, but it wouldn't be any faster than PENDING if it grows large since it is still an index. We could also delete and recreate PENDING after recovery if it is empty but very large, or even if non-empty by moving remaining files to the new PENDING (though this adds complexity).</p> <p>Andreas, do we already have a support for such a forwarding on the client side?</p> <p>big PENDING/ affects regular unlink/close while a separate index does not.</p> <p>I think a few smaller patches could be made to improve things here:</p> <ul class="alternate" type="square"> <li>scan of PENDING periodically (hourly?) to free objects that are not open but older than an hour (maybe /proc tunable to adjust scan interval), or if free inodes are getting low</li> <li>change recovery to keep redirection objects in PENDING after reboot</li> <li>implementing <a href="https://jira.whamcloud.com/browse/LU-7607" title="Preserve inode number after MDT migration " class="issue-link" data-issue-key="LU-7607">LU-7607</a> would also reduce the need to keep redirector inodes around, but not eliminate it for NFS filesystems unless the Lustre file handle could locate the redirected inode. I don't know if we would be able to detect if NFS is ever used on an object, since this is local to the client, but migration may happen on another client. I guess it would be possible for the client to send a flag/set_info to the MDS of the export operations are ever used. If not, then the redirector objects don't need to be kept much longer than the end of the migration transaction.</li> </ul> <p>I think that this functionality is being implemented by patch <a href="https://review.whamcloud.com/38135" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38135</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-11025" title="DNE3: directory restripe" class="issue-link" data-issue-key="LU-11025"><del>LU-11025</del></a> dne: add FID mapping interfaces</tt>"?</p> Related LU-2430 LU-11025 LU-7607 Development Rank 1|hzy0gv: Rank (Obsolete) 9223372036854775807 Severity