[LU-18] Allow 100k open files on single client

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.11.0
• Affects Version/s: Lustre 2.0.0, Lustre 2.1.0
• Bugzilla ID: 24,217

Epic

interoperability performance

• Assignee: WC Triage
• Reporter: Niu Yawei (Inactive)

Description

Allow 100k open files per client. Fix client to not store committed open RPCs in the resend list but instead reopen files from the file handles upon recovery (see Simplified Interop) to avoid O behaviour when adding new RPCs to the RPCs-for-recovery list on the client. Fix MDS to store "mfd" in a hash table instead of a linked list to avoid O behaviour when searching for an open file handle. For debugging it would be useful to have a /proc entry on the MDS showing the open FIDs for each client export.

Issue Links

• is related to: LU-5703 Quiesce client mountpoints from the server
• is related to: LU-5964 test large number of concurrent open files on client
• Trackbacks: JIRA and Confluence Upgrade Plan Problem statement Currently our bug tracking system (JIRA) and wiki (Confluence) are running versions that are frighteningly outofdate, to the point that for our upgrade of Confluence,...

Comments

[Andreas Dilger]

The referenced patches have landed, and this is likely fixed. There doesn't seem to be any value keeping it open longer.

[Andreas Dilger]

Patch https://review.whamcloud.com/12885 "LU-5964 tests: open a large number of files at once" adds a test case for this, which has exposed some issues with memory usage when many files are open. That issue is being addressed by patch https://review.whamcloud.com/27208 "LU-9514 ptlrpc: free reply buffer for replay RPC".

While that patch reduces the memory usage of saved open replay RPC buffers, it would be better to fix the open replay code as described here - to regenerate a new RPC to reopen files after the initial create has committed, rather than saving the open RPC indefinitely. Saving the RPCs wastes memory, and makes recovery more complex because the RPC format cannot be changed if the server is upgraded when it is offline.

[Andreas Dilger]

It looks like there is a copy of the patch at https://bugzilla.lustre.org/show_bug.cgi?id=24217

[Niu Yawei (Inactive)]

Unfortunately, I can't find a local copy neither. (it could be lost when replacing laptop)

The most significant change in the patch (client side changes) has been merged in master along with the fix of LU-2613. The server side change is about not reusing open handle on server side when do open replay.

[Andreas Dilger]

I was trying to find the patch for this ticket, but it seems it was in the old "lustre" project (not "fs/lustre-release" used today) at http://review.whamcloud.com/171/ which has since been removed.

Niu, do you still have a copy of this patch that you could upload to fs/lustre-release?