[LU-18192] TBF: generic combination TBF types with different granularities

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Severity: 3
• Assignee: Qian Yingjin
• Reporter: Qian Yingjin

Description

In the current NRS TBF design, a server classifies the incoming I/O requests into TBF buckets according to the NRS TBF rule with different rate limits.
The requests in each TBF bucket are scheduled in FIFO order and the RPC rate is limited per TBF bucket based.

Each TBF bucket is usually in unit of TBF types or the combination of various TBF types. i.e.
NID TBF. -> (nid)
JobID TBF -> (jobid)
UID TBF -> (uid)
GID TBF -> (gid)
opcode TBF -> (opcode)
generic TBF -> (nid, jobid, uid, gid, opcode)

The more general the scheduling type, the finer the granularity of the scheduling TBF bucket. It will generate much more TBF buckets for various scheduling classes.

Thus we can define more coarser TBF scheduling types according to the requirement of the users. i.e.

# lctl set_param ost.OSS.ost_io.nrs_policies="tbf nid+opcode"
or
# lctl set_param ost.OSS.ost_io.nrs_policies="tbf uid+gid+nid"

NID+Opcode TBF -> (nid, opcode)
UID+GID+NID TBF -> (uid, nid, gid)

Issue Links

• is related to: LU-18606 refcount_t warning when running sanityn/77j (refcount_t: addition on 0; use-after-free.)
• is related to: LU-18400 panic_on_warn etc. in test-framework.sh
• is related to: LU-8130 Migrate from libcfs hash to rhashtable
• is related to: LU-17503 IO500: improve NRS TBF to sort requests by object offset for ior-hard-write

Comments

[Gerrit Updater]

"Qian Yingjin <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58143
Subject: LU-18192 nrs: cleanup jobid/opcode/uid/gid NRS TBF types
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 504a433be334618d877336250152fa333968e00b

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56227/
Subject: LU-18192 nrs: move nid/uid/gid/opcode/jobid into generic key
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 7955578d771ceb1ec01388f9daaa51b9ea03d7bb

[Gerrit Updater]

"Qian Yingjin <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56237
Subject: LU-18192 nrs: TBF combination with different granularities
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 074c9b671e47bec407ff1bb43bd3083830b59c20

[Gerrit Updater]

"Qian Yingjin <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/56227
Subject: LU-18192 nrs: move nid/uid/gid/opcode/jobid into generic key
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: fe69bc0363ba466647a940344418603a19dd8b4d