Details
-
New Feature
-
Resolution: Unresolved
-
Minor
-
None
-
None
-
3
-
9223372036854775807
Description
At present if there are many agents available for the coordinator to send HSM work to, the coordinator will send as many actions as it can to the first agent before moving onto the next one. This can result in pretty swamped singular agents while others remain idle.
This feature adds a new tunable, max_hal_count, which governs how many hsm action list entries can be sent in a given request to an agent. It also tracks when requests were sent to a specific agent last, enabling the coordinator to round robin over the pool of agents easily and better load-balance than it was able to do before.
Patch to be sent shortly.
Attachments
Activity
"Ellis Wilson <elliswilson@microsoft.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/54757
Subject: LU-17738 hsm: Enable CDT to load balance to agents more evenly
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: e7665111f1a811c928f70428c9915c3f44a818ee
This policy should be selectable. If the agent is writing directly to a tape device (which AFAIK some systems do, maybe FNAL?) then it makes sense to keep that tape drive busy before starting on the next tape. I think this is the exception rather than the rule (most HSM systems write to a staging disk pool to aggregate files before moving them to tape.
alex.ku, alex, do you have any comment on this? IIRC you were previously using HSM in this manner.