https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13151 Lustre <p>Some tools that maintain copies of filesystem metadata in an external database monitor changelogs for changes to the filesystem, in order to determine files that need to be rescanned. This can result in a large number of small updates to the external database that can reduce the tool's ingest performance. It might be beneficial to instead track and scan complete directories that contain modified files and update the external database using bulk operations. Adding the parent FID to MTIME changelogs allows to more efficiently determine the parent directories for some types of file data modifications, by issuing OBD_IOC_FID2PATH once for each parent FID, instead of once for each file FID.</p> LU-13151

Add parent FID to MTIME changelogs

Improvement Minor Resolved Fixed Nikitas Angelinas Nikitas Angelinas Fri, 17 Jan 2020 11:06:01 +0000 Fri, 13 Jan 2023 14:00:40 +0000 Tue, 8 Sep 2020 19:01:58 +0000 Lustre 2.13.0 Lustre 2.14.0 0 3 <p>Nikitas Angelinas (nangelinas@cray.com) uploaded a new patch: <a href="https://review.whamcloud.com/37264" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37264</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13151" title="Add parent FID to MTIME changelogs" class="issue-link" data-issue-key="LU-13151"><del>LU-13151</del></a> mdt: add parent FID to MTIME changelogs<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 120166579c678d8fa6610b192fc369d447131140</p> <p>The patch submitted uses the link EA to add the parent FID in the MDT; we have not verified whether this affects performance negatively. We think this patch could help in detecting some types of file data modifications, but not all, as MTIME changelogs don't seem to be generated e.g. when appending to a file.</p> <p>In particular, code like <a href="https://github.com/mar-file-system/GUFI" class="external-link" target="_blank" rel="nofollow noopener">GUFI</a> can take a "suspects" list of modified directories to rescan. Without the parent fid in the changelog, each mtime record needs a fid2path to find the parent directory to rescan. With the parent in the changelog, all the information except the parent can be thrown away, and all records "collapse" into a small set of modified directories. </p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37264/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37264/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13151" title="Add parent FID to MTIME changelogs" class="issue-link" data-issue-key="LU-13151"><del>LU-13151</del></a> mdt: add parent FID to Changelog recordss<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b9d7f8e07ad68a51b4fcc09babe48a68fcbd6041</p> <p>Landed for 2.14</p> Related LU-16469 Development Rank 1|i00s6v: Rank (Obsolete) 9223372036854775807