Details
-
New Feature
-
Resolution: Unresolved
-
Minor
-
None
-
None
-
3
-
9223372036854775807
Description
Nowadays, this is not possible to add several permanents TBF rules via "lctl set_param -P".
The MGS will erase the old value of a parameter in the llog "CONFIG/params".
For example if we want 2 add the following rules:
# lctl set_param -P ost.OSS.ost_io.nrs_policies="tbf jobid"
# lctl set_param -P ost.OSS.ost_io.nrs_tbf_rule="start login jobid={*login*} rate=5000"
# lctl set_param -P ost.OSS.ost_io.nrs_tbf_rule="start rbh jobid={*rbh*} rate=100000"
# lctl get_param ost.OSS.ost_io.nrs_tbf_rule
ost.OSS.ost_io.nrs_tbf_rule=
regular_requests:
CPT 0:
rbh {*rbh*} 100000, ref 0
login {*login*} 5000, ref 0
...
# lctl llog_print params | grep tbf
- { index: 14, event: set_param, device: general, parameter: ost.OSS.ost_io.nrs_policies, value: tbf jobid }
- { index: 20, event: set_param, device: general, parameter: ost.OSS.ost_io.nrs_tbf_rule, value: start rbh jobid={*rbh*} rate=100000 }
Only the last one is set in the llog config.
So, if we unmount/remount the OST:
# lctl get_param ost.OSS.ost_io.nrs_tbf_rule
ost.OSS.ost_io.nrs_tbf_rule=
regular_requests:
CPT 0:
rbh {*rbh*} 100000, ref 0
default {*} 10000, ref 0
For now, at the CEA, we use udev rules to automatically set permanent tbf rules. But this is not really clean (hard sleep to wait for lustre startup etc...).
I see several way to fix that issue:
- The simplest way would be to add "--append" option to "set_param -P". This would advise the MGS to not remove the old entries. But this will not be easy for admin to maintain those entries: if he want to remove one rule at startup, he has to resent all the rules. Then, to overcome that issue, we could extend "lctl set_param -P -d" to match value too.
- We can also add a special cases to handle "set_param -P" for "nrs_tbf_rules". And we can manage this like for OST pools.
- We can also make all TBF rules permanent and add specific ldiskfs files to store those rules for each target (iam files ?). But this seems complex.
