[LU-11407] Improve stats data

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0, Lustre 2.15.2
• Assignee: Andreas Dilger
• Reporter: Andreas Dilger

Description

It would be useful to store and report the "job start" time for the JobStats. Currently we show in the obdfilter.*.job_stats file:

- job_id:          mythbackend.0
  snapshot_time:   1537384753
  read_bytes:      { samples:         321, unit: bytes, min:    4096, max: 4194304, sum:      1025404928 }
  write_bytes:     { samples:       12656, unit: bytes, min:   22028, max:  919476, sum:      5413800656 }
  sync:            { samples:       11168, unit:  reqs }
  statfs:          { samples:       31249, unit:  reqs }

but this doesn't tell us anything about when this job started, so we can't find the throughput or IOPS rates. It should be simple to store the first time this job reported IO so that we can have some idea about the rate.

A further enhancement would be to store the full brw_stats into the job_stats file, but that is a more complex change.

Issue Links

• is related to: LU-16231 Lustre stats header incorrectly using boot time
• is related to: LU-15826 jobstats output can produce invalid yaml ( part 2 )
• is related to: LU-13123 Add list of client NIDs to job_stats output
• is related to: LU-18359 Do not escape "..." around jobid_name
• is related to: LU-12872 Adding more stats into JOBSTATS
• is related to: LU-16555 print more YAML compatible special chars for jobid
• is related to: LU-16599 clearing jobstats should match output format
• is related to: LU-17005 allow lprocfs stats header to be disabled
• is related to: LU-13597 add processing time/latency, IO sizes, stats to job_stats

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/50734/
Subject: LU-11407 obdclass: init osc.*.rpc_stats start_time
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 0176531449899c30ebdeaf372464fd0685ca3645

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/50734
Subject: LU-11407 obdclass: init osc.*.rpc_stats start_time
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: f8511d3f4ed5a7a2b426383f036208ec64de1cf5

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/48539/
Subject: LU-11407 tgt: cleanup job_stats output printing
Project: fs/lustre-release
Branch: b2_15
Current Patch Set:
Commit: 56c0d6316fbf29eac019f5a7c823199592027b25

[Gerrit Updater]

"Jian Yu <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/48539
Subject: LU-11407 tgt: cleanup job_stats output printing
Project: fs/lustre-release
Branch: b2_15
Current Patch Set: 1
Commit: 8eea90f503a35942c8af25520d6485827f9370f3

[Peter Jones]

Landed for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/37764/
Subject: LU-11407 tgt: cleanup job_stats output printing
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 338381574b643da064e90e44d75be85d1be3a93c

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/33201/
Subject: LU-11407 obdclass: add start time to stats files
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ea2cd3af7bfabfa6876727ee44495f4c331bea8e

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37764
Subject: LU-11407 tgt: cleanup job_stats output printing
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: cf1e6bb6403ff3114976c3c07e1aa65ab9230db3

[Li Xi]

I don't think floating-point math is necessary, since 64 bit interger should be enough for most of the collectors. A rate with high precision doesn't help too much for analysis.

Anyway, the elapsed_time helps a lot. Thanks.

[Joe Grund]

No issue on my end, just want to know where I need to target.