[LU-8975] Batching time space for TBF scheduling

Thu, 16 Aug 2018 21:16:13 +0000

In our actual TBF implementation, tokens in a bucket are updated according to the elapsed time and token rate when the class queue is actually able to dequeue an RPC request. Our TBF uses a global timer with expiration time always setting to the latest deadline of classes (bucket) and all classes are sorted according to their deadlines. When the timer expired, the class with smallest deadline will be selected. Then the first RPC request in this class queue will be dequeued and handled by an idle service thread.

Current TBF assigns class deadline spaced by increments of 1/r to successive requests in the class queue(where r is RPC rate of the class), If all requests are scheduled in order of their deadline values, the class (application) will receive service in proportion to r.

class.deadline = class.last_check_time + 1/r;

Since each time the timer expired, it usually only generates one token and serves one request for the selected class with smallest deadline, this may destroy the sequentiality and locality of I/O requests in the class queue, resulting performance regression. i.e. two classes (jobs) may submit IO reuqests to the underlying disk system interleavely by using TBF, it is not conductive to the disk scheduler to optimize.

A batching the time space strategy is proposed. Under this strategy,

class.deadline = class.last_check_time + batch_num * 1/r;

By this way, when the global timer expired, the selected class will accumulate at less batch_num tokens. Thus batch_num _requests in the class queue can be handled batchly until exhaused _batch_num tokens.

This stragegy not only considers the sequentiality and locality of requests, but also guarantee the rate limit.

The format of command to configure the batch_num for a rule is shown as follow:

start RuleName jobid={dd.0} batch_num=32 rate=320

Whamcloud Community JIRA

[LU-8975] Batching time space for TBF scheduling