https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2713 Lustre <p>The client-side HSM coordinator patches in <a href="http://review.whamcloud.com/5029" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5029</a> and <a href="http://review.whamcloud.com/5030" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5030</a> were landed, but Oleg realized that there are no client-side limits on the number of concurrent RPCs that can be sent.</p> <p>This could potentially overwhelm the MDS service threads and block all other requests if they become blocked handling HSM requests, or if they are not being processed very quickly.</p> <p>Please institue a client-side RPC limit, like cl_max_rpcs_in_flight, but for HSM requests, that introduces some reasonable limit.</p> <p>The ticket is assigned to Jinshan, but only because we cannot currently assign it to someone external.</p> LU-2713

limit HSM RPC count from client

Technical task LU-2061 Blocker Resolved Fixed John Hammond Andreas Dilger MB Wed, 30 Jan 2013 18:00:08 +0000 Wed, 13 Mar 2013 08:57:16 +0000 Wed, 13 Mar 2013 08:57:16 +0000 Lustre 2.4.0 Lustre 2.4.0 0 7 <p>I will work on a patch</p> <p>Can the client side maximum include 0 as a possible value (or even a default, unless root)? Otherwise, a malicious/accident-prone user can simply issue HSM RPCs from multiple clients: "Hmm, login1 seems wedged. I think I'll kill my ssh session and try this again on login2."</p> <p>Would an MDT side limit be better?</p> <p>As I understand the limit comes from the MDT capacity to receive RPC request, so an MDT side is better but if the MDT had to count the requests it will already have received them so too late. The client side is a simple way to limit the load.</p> <p>Do you confirm you work on a patch (so I will not prepare one) </p> <p>I was proposing that the MDT keep a semaphore (as with cl_max_rpcs_in_flight) but that it do a non blocking down. If the semaphore would block then it returns -EAGAIN to the client. Then the client must wait and retry.</p> <p>I understood that processing some HSM requests would put the MDT thread to sleep until the coordinator responded. Is that correct? I have only seen the stubbed out version of mdt_hsm.c. Will any of these handlers every have to wait for tape?</p> <p>In either case (waiting on the coordinator or waiting on tape) I think it must be handled as an unbounded wait by Lustre.</p> <p>I confirm that I will work on a patch.</p> <p>HSM request are not blocking, they just record something to do on the MDT and the restore/archive is done asynchronously by coordinator. We the use of EAGAIN the only risk is to have slow clients which are never served because fast one are always taking the slots. We need a way to be sure all the clients are doing progress in their call list</p> <p>John, the current RPC throttling mechanism for OSC and MDC RPCs is on the client. While this is not ideal, the problem is indeed that if the server has seen the request that it is too late to throttle it.</p> <p>At this stage, we're just looking for an equivalent to max_rpcs_in_flight for the HSM requests, so they do not overwhelm the server.</p> <p>OK, thanks for the clarification.</p> <p>Please see <a href="http://review.whamcloud.com/5616" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/5616</a>.</p> <p>Landed for 2.4</p> Related LU-2949 Development Rank 1|hzvi9j: Rank (Obsolete) 6608