[LU-9228] Hard TBF Token Compensation under congestion

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.11.0
• Labels: patch
• Assignee: Qian Yingjin (Inactive)
• Reporter: Qian Yingjin (Inactive)

Description

During TBF evaluation, we find that when the sum of I/O bandwidth requirements for all classes exceeds the system capacity, the classes with same rate limits get less bandwidth than preconfigured evenly.
The reason is as follow. Under heavy load on a congested server, it will result in some missed deadlines for some classes. The calculated tokens may larger than 1 during dequeuing. In the original implementation, all classes are equally handled to simply discard exceeding tokens.
Thus, a Hard Token Compensation (HTC) strategy is proposed. A class can be configured with HTC feature by the rule it matches. This feature means that requests in this kind of class queues have high real-time requirements and that the bandwidth assignment must be satisfied as good as possible. When deadline misses happen, the class keeps the deadline unchanged and the time residue (the remainder of elapsed time divided by 1/r) is compensated to the next round. This ensures that the next idle I/O thread will always select this class to serve until all accumulated exceeding tokens are handled or there are no pending requests in the class queue.

A new command format is added to enable HTC feature for a rule:

start $ruleName jobid={dd.0} rate=100 realtime=1

Issue Links

• is related to: LU-3558 NRS TBF policy for QoS purposes
• is related to: LUDOC-328 documentation updates for complex TBF policies

Comments

[Emoly Liu]

The documentation update of HTC strategy is at https://review.whamcloud.com/31628 .

[Peter Jones]

Landed for 2.11

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26087/
Subject: LU-9228 nrs: TBF realtime policies under congestion
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d11fa2c279593634cf6c4196b413a6d285b24e10

[Gerrit Updater]

Yingjin Qian ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26087
Subject: LU-9228 nrs: Hard Token Compensation under congestion
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 6860deca4bd4e5efcc5a68b44beb4eb9bd7df95b