https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12332 Lustre <p><tt>IOC_MDC_GETFILEINFO</tt> is a very convenient ioctl to retrieve metadata information (stat + lov EA v1) from MDT. This avoid going to OSTs if you don't need information stored over there (like file size).</p> <p>This is wrapped by :</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">  <span class="code-object">int</span> get_lmd_info_fd(<span class="code-object">char</span> *path, <span class="code-object">int</span> parent_fd, <span class="code-object">int</span> dir_fd,       void *lmdbuf, <span class="code-object">int</span> lmdlen, <span class="code-keyword">enum</span> get_lmd_info_type type) </pre> </div></div> <p>but this function is not exported and it could be nice to also add a llapi prefix.</p> LU-12332

Add a liblustreapi call for IOC_MDC_GETFILEINFO

Improvement Minor Resolved Fixed WC Triage Aurelien Degremont Thu, 23 May 2019 09:16:55 +0000 Tue, 28 Jul 2020 23:46:24 +0000 Tue, 28 Jul 2020 23:46:24 +0000 Lustre 2.13.0 Lustre 2.14.0 Lustre 2.12.4 0 3 <p>Aurelien, thanks for filing this ticket, it is something that I've been meaning to do as well.</p> <p>The patch <a href="https://review.whamcloud.com/33545" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/33545</a> adds functionality to <tt>IOC_MDC_GETFILEINFO</tt> so that it can optionally return the Lazy Size-on-MDT data from the initial MDS RPC, so that tools such as "<tt>lfs find"</tt> and other scanners can use this as the approximate file size, instead of having to do an extra <tt>stat()</tt> on the file each time to get the size/blocks.</p> <p>As such, I'd like to plumb in the <tt>llapi_&#42;</tt> interface for this command to include the "<tt>__u64 lmd_flags</tt>" an argument, and separate out the pointer to "<tt>struct stat</tt>" from "<tt>struct lov_user_md</tt>" so that it is isolated from the underlying ioctl interface a bit. That allows the internals to call either <tt>IOC_MDC_GETFILEINFO</tt> or <tt>IOC_MDC_GETFILEINFO_OLD</tt> depending on which command the kernel supports without having to expose this to userspace. As such, I don't think that just renaming <tt>get_lmd_info_fd()</tt> is the right way to go.</p> <p>Actually I was also thinking about add a call which was not exactly <tt>get_lmd_info_fd()</tt> but take this opportunity to improve that a little bit. Your proposal makes sense.</p> <p>Enabling this call to return more information in one RPC is interesting. Tools like Robinhood can benefit from a call which "give me everything in 1 RPC", like stats, layout, ...</p> <p>So, whatever this call will return with 1 RPC, the more, the better</p> <p>This ticket seems to be addressed by <a href="https://jira.whamcloud.com/browse/LU-11367" title="integrate LSOM with lfs find" class="issue-link" data-issue-key="LU-11367"><del>LU-11367</del></a> patch<br/> <a href="https://review.whamcloud.com/#/c/35167/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/35167/</a></p> <p>int llapi_get_lum_file_fd(int dir_fd, const char *fname, __u64 *valid,<br/> lstatx_t *statx, struct lov_user_md *lum,<br/> size_t lumsize);</p> <p>I don't want to hold up the landing of <a href="https://jira.whamcloud.com/browse/LU-11367" title="integrate LSOM with lfs find" class="issue-link" data-issue-key="LU-11367"><del>LU-11367</del></a>, but it's not quite ideal yet.</p> <p>llapi_get_lum_file_fd requires an open parent directory FD to send the ioctl to. It looks like this has to be the direct parent given the strrchr call in get_lmd_info_fd, rather than a true "path relative to the parent fd", which might allow us to issue the ioctl on the FS root in all cases, and avoid opening the parent dirs. </p> <p>Since both stat() and getfattr() can be issued against the path with no opens, it seems like the info should be obtainable with even fewer RPCs. For example, llapi_get_lum_file could, instead of opening the parent dir and calling get_lum_file_fd, just call stat and getfattr lustre.lov, and avoid the open. Of course, stat and getfattr are two calls - but in reality we know that the stat() has already retrieved the layout info (because it needs to talk to the OSTs for size). So this is my long-winded way of asking: why can't we get all the info we want for the cost of a single stat()?</p> <p>&gt; why can't we get all the info we want for the cost of a single stat()?</p> <p> Because stat() will also send RPCs to OSTs and this is what we want to avoid, no?</p> <p>&gt; Because stat() will also send RPCs to OSTs and this is what we want to avoid, no?<br/> Perhaps better phrased as "the cost of a single RPC". My point is that a stat() call gets all the metadata that the MDS holds, both the MD in the inode and the layout, for the cost of a single ldlm_ibits_enqueue call to the MDS. At that point, all the info (except size) is available on the client, without an "open".</p> <p>If we filled in the stat size with lazy some data, and interrupted the stat on the client before it tried to get the size from the OSTs, we would have everything that llapi_get_lum_file_fd has, with a single RPC.</p> <p>Having said all that:<br/> &gt;RPCs to OSTs and this is what we want to avoid<br/> Interestingly: no. That has always been the assumption, that "stats are slow because they have to go to OSTs". But that doesn't actually seem to be the case. We can get very high stat rates, even going to OSTs. The problem really seems to be RPCs to the MDS, and reducing the MDS RPC count is actually the win. In particular, directory open on MDS is slow, maybe because of locks, so it's likely (although I haven't tested) that the ioctl is slower that the stat, even including OST RPCs. </p> <p>The new llapi was added as part of <a href="https://jira.whamcloud.com/browse/LU-11367" title="integrate LSOM with lfs find" class="issue-link" data-issue-key="LU-11367"><del>LU-11367</del></a> (in 2.13.0 and 2.12.4).</p> <p>The ability to selectively get file attributes (without doing an OST RPC) is added with the <tt>statx()</tt> API in <a href="https://jira.whamcloud.com/browse/LU-10934" title="integrate statx() API with Lustre" class="issue-link" data-issue-key="LU-10934"><del>LU-10934</del></a> in 2.14.</p> Related LU-11367 LU-10934 LU-11695 Development Rank 1|i00gu7: Rank (Obsolete) 9223372036854775807