[LU-14974] Improve the repartition of requests between copytools

Mon, 30 Aug 2021 12:41:00 +0000

Currently, when reading new requests, the HSM coordinator builds *hsm_action_list*s ** by first trying to find a suitable list in the already constructed lists (i.e. with the same archive_id and enough space meaning less than LDLM_MAXREQSIZE). Then the request is added in the list if possible or a new list is allocated for it.

The issue with this approach is that if the number of requests is not large enough to fill an entire list, all the requests will be sent to one copytool only, leaving the other idles. If these few requests are concerning "big" files, this will create a huge load on one copytool and waste a lot of time.

The idea would be to build more *hsm_action_list*s based on the number of copytools per archive_id.

If we have, say, 3 copytools for the archive_id 1, we can build 3 lists and distribute the requests of archive_id 1 evenly in them. These lists will then be more likely to be sent to the 3 copytools depending on their current load. Once the 3 lists are full, we can allocate 3 new ones and continue the process.

Whamcloud Community JIRA

[LU-14974] Improve the repartition of requests between copytools