[LU-11968] tbf QOS gid rules not being enforced

• Type: Bug
• Resolution: Not a Bug
• Priority: Major
• Affects Version/s: Lustre 2.12.0
• Severity: 3
• Assignee: Li Xi
• Reporter: Mahmoud Hanafi

Description

Setting up tbf qos for ost_io like this

lctl set_param ost.OSS.ost_io.nrs_policies="tbf gid"
lctl set_param ost.OSS.ost_io.nrs_tbf_rule="start viz gid={1128} rate=10000"
lctl set_param ost.OSS.ost_io.nrs_tbf_rule="start css gid={1125} rate=10000"
lctl set_param ost.OSS.ost_io.nrs_tbf_rule="change default rate=100"

Running IOR as a user with group css, the writes are correctly throttled by the css rule. But read are throttled by the default rate.

Lowering the default rate decrease reads bw, and increasing the default rate increases read bw.

IOR is getting 8GB/sec Writes and 1.4GB/sec reads. It should be 8GB/sec for read and writes.

 srv2 /sys/kernel/debug/lustre/ost/OSS/ost_io # cat nrs_tbf_rule
regular_requests:
CPT 0:
css {1125} 10000, ref 1
viz {1128} 10000, ref 0
default {*} 100, ref 1
CPT 1:
css {1125} 10000, ref 1
viz {1128} 10000, ref 0
default {*} 100, ref 1
CPT 2:
css {1125} 10000, ref 1
viz {1128} 10000, ref 0
default {*} 100, ref 1
CPT 3:
css {1125} 10000, ref 1
viz {1128} 10000, ref 0
default {*} 100, ref 1
CPT 4:
css {1125} 10000, ref 1
viz {1128} 10000, ref 0
default {*} 100, ref 1