Vitaliy, in my previous investigation of a similar issue in LU-14144 I couldn't find any good reason in the code or commit history why max_mod_rpcs_per_client was specifically a module parameter on the server and not a regular sysfs parameter. There doesn't appear to be any runtime dependency on this value (i.e. it doesn't define a static number of slots for the per-client replies or anything), and the only thing it is used for is to pass the limit to the client. For the same reason, there also doesn't appear to be a particularly hard limitation why the client cannot change and exceed the server-provided parameter, except to avoid overloading the server with too many RPCs at once, but that may also be true of the current limit with a larger number of clients, no different than "max_rpcs_in_flight".
It seems reasonable to add a per-MDT "max_mod_rpcs_in_flight" tunable parameter to lustre/mdt/mdt_lproc.c so that it can be set with "lctl set_param" at runtime, for example like async_commit_count. The global max_mod_rpcs_per_client parameter should be used as the initial value, and add "(deprecated)" to the module description in mdt_handler.c.
Mahmoud, the console error message printed when the client limit is reached is "myth-MDT0000-mdc-ffff979380fc1800: can't set max_mod_rpcs_in_flight=32 higher than ocd_maxmodrpcs=8 returned by the server at connection" but I agree this isn't totally clear. Instead of reporting "ocd_maxmodrpcs" (which is an internal field name) it should report the new "mdt.myth-MDT0000.max_mod_rpcs_in_flight" parameter, which would steer the admin to the right location to change this value. However, in the current implementation it would still be necessary to unmount/remount (or at least force a client reconnection) if this parameter is changed.
The main question is whether there is any value for the MDS to "limit" the value that can be set by the client (which is not done for max_rpcs_in_flight or most other parameters) or if the client should be able set this larger than the default value the MDT returned (maybe some upper limit like 4x or 8x the MDT limit)? That would allow something like "lctl set_param -P ..max_mod_rpcs_in_time" to affect both the clients and servers.
Minor fix for limit in
LU-16558