[LU-18351] Job stats scaling

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.17.0
• Severity: 3
• Assignee: Shaun Tancheff
• Reporter: Shaun Tancheff

Description

When jobs reach very large numbers > 100k

During walking the list of jobs (ojs_list) and restarting is frequent due to reallocation and the pos requested for restart may not be the last_job.

Save every n'th pos (here it is 512) so the max walk on restart is 512.

Issue Links

• is related to: LU-18610 Avoid double putting job ref
• is related to: LU-8130 Migrate from libcfs hash to rhashtable