[LU-11407] Improve stats data

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0, Lustre 2.15.2
• Assignee: Andreas Dilger
• Reporter: Andreas Dilger

Description

It would be useful to store and report the "job start" time for the JobStats. Currently we show in the obdfilter.*.job_stats file:

- job_id:          mythbackend.0
  snapshot_time:   1537384753
  read_bytes:      { samples:         321, unit: bytes, min:    4096, max: 4194304, sum:      1025404928 }
  write_bytes:     { samples:       12656, unit: bytes, min:   22028, max:  919476, sum:      5413800656 }
  sync:            { samples:       11168, unit:  reqs }
  statfs:          { samples:       31249, unit:  reqs }

but this doesn't tell us anything about when this job started, so we can't find the throughput or IOPS rates. It should be simple to store the first time this job reported IO so that we can have some idea about the rate.

A further enhancement would be to store the full brw_stats into the job_stats file, but that is a more complex change.

Issue Links

• is related to: LU-16231 Lustre stats header incorrectly using boot time
• is related to: LU-15826 jobstats output can produce invalid yaml ( part 2 )
• is related to: LU-13123 Add list of client NIDs to job_stats output
• is related to: LU-18359 Do not escape "..." around jobid_name
• is related to: LU-12872 Adding more stats into JOBSTATS
• is related to: LU-16555 print more YAML compatible special chars for jobid
• is related to: LU-16599 clearing jobstats should match output format
• is related to: LU-17005 allow lprocfs stats header to be disabled
• is related to: LU-13597 add processing time/latency, IO sizes, stats to job_stats