https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-18 Lustre <p>Allow 100k open files per client. Fix client to not store committed open RPCs in the resend list but instead reopen files from the file handles upon recovery (see Simplified Interop) to avoid O<img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/thumbs_down.png" height="16" width="16" align="absmiddle" alt="" border="0"/> behaviour when adding new RPCs to the RPCs-for-recovery list on the client. Fix MDS to store "mfd" in a hash table instead of a linked list to avoid O<img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/thumbs_down.png" height="16" width="16" align="absmiddle" alt="" border="0"/> behaviour when searching for an open file handle. For debugging it would be useful to have a /proc entry on the MDS showing the open FIDs for each client export. </p> LU-18

Allow 100k open files on single client

Improvement Minor Resolved Fixed WC Triage Niu Yawei Mon, 22 Nov 2010 17:50:15 +0000 Tue, 11 May 2021 19:21:42 +0000 Tue, 11 May 2021 19:21:42 +0000 Lustre 2.0.0 Lustre 2.1.0 Lustre 2.11.0 Fri, 10 Dec 2010 00:00:00 +0000 0 4 <p>Talked with Andreas and Ericm, to avoid the conflicts with the simplified interop work, also for easy patch/feature management, I decided to use the separate list for the committed open on client (as Andreas suggested) at the first stage.</p> <p>For the server side mfd list, I found that in normal operations, the mfd can always be found in the general handle hash table(class_handle_hash), the list only be scanned in following two cases:</p> <ul class="alternate" type="square"> <li>For the resent open(and setattr in som), search the mfd in list by matching xid;</li> <li>For the replayed close(and setattr/done_writing in som), search the mfd in list by matching mfd_old_handle (I don't quite understand this, why can't we just keep the old handle for the replayed open? thus this mfd_old_handle trick will be gone);</li> </ul> <p>so I suppose what we want is:</p> <ul class="alternate" type="square"> <li>Store the mfd in cfs_hash in stead of global handle hash table (indexed by handle), which requires modifing the general handle hash code to export a handle generator function.</li> <li>Keep old handle for the replayed open, thus the mfd_old_handle matching work can be avoid.</li> <li>Create another cfs_hash for the mfd, indexed by xid, thus list searching for resent open can be avoid.</li> </ul> <p>Have exchanged my ideas with Andreas.</p> <p>Have submit the patch for review.</p> <p>Updated patch according to reviewers' comments, submitted for the 2nd round review.</p> <p>I was trying to find the patch for this ticket, but it seems it was in the old "lustre" project (not "fs/lustre-release" used today) at <a href="http://review.whamcloud.com/171/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/171/</a> which has since been removed.</p> <p>Niu, do you still have a copy of this patch that you could upload to fs/lustre-release?</p> <p>Unfortunately, I can't find a local copy neither. (it could be lost when replacing laptop)</p> <p>The most significant change in the patch (client side changes) has been merged in master along with the fix of <a href="https://jira.whamcloud.com/browse/LU-2613" title="opening and closing file can generate &#39;unreclaimable slab&#39; space" class="issue-link" data-issue-key="LU-2613"><del>LU-2613</del></a>. The server side change is about not reusing open handle on server side when do open replay.</p> <p>It looks like there is a copy of the patch at <a href="https://bugzilla.lustre.org/show_bug.cgi?id=24217" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/show_bug.cgi?id=24217</a></p> <p>Patch <a href="https://review.whamcloud.com/12885" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/12885</a> "<a href="https://jira.whamcloud.com/browse/LU-5964" title="test large number of concurrent open files on client" class="issue-link" data-issue-key="LU-5964"><del>LU-5964</del></a> tests: open a large number of files at once" adds a test case for this, which has exposed some issues with memory usage when many files are open. That issue is being addressed by patch <a href="https://review.whamcloud.com/27208" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27208</a> "<a href="https://jira.whamcloud.com/browse/LU-9514" title="sanity test 51f times out due to out of memory" class="issue-link" data-issue-key="LU-9514"><del>LU-9514</del></a> ptlrpc: free reply buffer for replay RPC".</p> <p>While that patch reduces the memory usage of saved open replay RPC buffers, it would be better to fix the open replay code as described here - to regenerate a new RPC to reopen files after the initial create has committed, rather than saving the open RPC indefinitely. Saving the RPCs wastes memory, and makes recovery more complex because the RPC format cannot be changed if the server is upgraded when it is offline.</p> <p>The referenced patches have landed, and this is likely fixed. There doesn't seem to be any value keeping it open longer.</p> Related LU-5703 LU-5964 Bugzilla ID 24217.0 Development Epic interoperability performance Rank 1|hzw17b: Rank (Obsolete) 10276