[LU-13857] jobstats output can produce invalid yaml

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.15.0
• Severity: 3
• Assignee: Feng Lei
• Reporter: Joe Grund

Description

When calling lctl get_param obdfilter.*OST*.job_stats there may be values that fill all left padded space. For example:

 

- job_id:          56796601
  snapshot_time:   1596652184
  read_bytes:      { samples:       62829, unit: bytes, min:    4096, max:16777216, sum:    141697982464 }
  write_bytes:     { samples:        2146, unit: bytes, min:     955, max: 1154584, sum:       320832894 }
  getattr:         { samples:           0, unit:  reqs }
  setattr:         { samples:           0, unit:  reqs }
  punch:           { samples:           1, unit:  reqs }
  sync:            { samples:           0, unit:  reqs }
  destroy:         { samples:           0, unit:  reqs }
  create:          { samples:           0, unit:  reqs }
  statfs:          { samples:           0, unit:  reqs }
  get_info:        { samples:           0, unit:  reqs }
  set_info:        { samples:           0, unit:  reqs }
  quotactl:        { samples:           0, unit:  reqs } 

The max value (max:16777216) has no space between the key and value and is now considered a scalar instead of a key value mapping.

Issue Links

• is related to: LU-15826 jobstats output can produce invalid yaml ( part 2 )

Comments

[Peter Jones]

Landed for 2.15

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/41709/
Subject: LU-13857 obdclass: Add white space to output valid YAML.
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 151f5322d30ec52a1b99c852e5adbdbbe6fc7e08

[Feng Lei]

https://review.whamcloud.com/#/c/41709/

[Peter Jones]

Feng Lei

Could you please investigate this issue?

Thanks

Peter

[James A Simmons]

Ah it the lack of spacing. Another thing I noticed is the indentation in job_stats which produces a different sequence of YAML tokens.

job_stats:

   - job_id:

produces 

STREAM START

[Block mapping]

 (Key token)

scalar job_stats

(Value token)

<b>Start Block (Sequence)</b>

 <b>Start Block (Entry)</b>

 [Block mapping]

(Key token)

scalar job_id

(Value token)

scalar 56744

While -----

job_stats:

• job_id: 1231

will drop the block sequence and well as the block end tokens.

produces 

[Joe Grund]

According to the spec max:16777216 is a scalar value and not a key / value pair.

https://yaml.org/spec/1.2/spec.html

Normally, YAML insists the “:” mapping value indicator be separated from the value by white space. A benefit of this restriction is that the “:” character can be used inside plain scalars, as long as it is not followed by white space. This allows for unquoted URLs and timestamps. It is also a potential source for confusion as “a:1” is a plain scalar and not a key: value pair.

[James A Simmons]

What exactly reports this as a incorrect YAML format? I wonder if this is a issue of the indentation since it generates different type of tokens. 

[Peter Jones]

Emoly

Could you please investigate?

Thanks

Peter