[LU-16007] NRS Jobid default RPC aggregation

Related — Fri, 17 Nov 2023 16:39:19 +0000

It would be useful to have a default NRS TBF rule that batched IOs on the servers based on their JobID, in a manner similar to how ORR or CRR batch IOs based on the Object ID or client NID.

Aggregating and completing IOs from a specific job is beneficial for aggregate IO throughput optimization. While this is locally unfair for one job to be prioritized over another, it avoids both jobs from being slowed down while their IOs are competing with each other and being interleaved "fairly" to the storage. Prioritizing any one JobID would at least allow it to finish its IO first, and then get on with computation (presumably no longer generating IO) and the other JobIDs can complete their IO with less contention. The IO completion time would probably be comparable for the last JobID, but may even be improved if the reduction in contention allows the IO to be more efficient (i.e. read- or write-only workload vs. mixed read/write from multiple jobs).

An implementation challenge would be to ensure that the same JobID is prioritized across all MDTs/OSTs, so that one job actually finishes its IO first, and does not have uneven completion time across targets. Self-balancing systems might do something (arbitrary) like prioritize IO based on lower JobID name, since this can be determined uniformly across targets without any central control. De-prioritized jobs would get a "credit" for a later priority boost (e.g. GIFT: A Coupon Based Throttle-and-Reward Mechanism for Fair and Efficient I/O Bandwidth Management on Parallel Storage Systems) so that overall the IO is fair and "cp" does not always have priority over "dd" if the JobID is "procname_uid".

Whamcloud Community JIRA

[LU-16007] NRS Jobid default RPC aggregation