[LU-17512] add conditional operator for 'jobid_name'

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Labels: easy lug24dd
• Severity: 3
• Assignee: Max Dilger
• Reporter: Andreas Dilger

Description

It would be useful to add a conditional operator like jobid_name=%j?%H:%e:%u in jobid_interpret_string() to allow using jobid_var if it is set, otherwise use the short hostname.

This allows adding the hostname for interactive jobs that don't have the jobid_var environment variable set (e.g. on login nodes or commands that are run on compute nodes outside of the configured job scheduler), while also not splitting up the job stats for a single 5000-node job into 5000x separate JobIDs to track on the servers for cases where jobid_var is set.

Issue Links

• is related to: LU-17296 NRS TBF default rules
• is related to: LU-14339 Add a option %H to avoid the jobid too long
• is related to: LU-16376 lprocfs_job_stats_log() Invalid jobid size (37), expect(32)
• is related to: LU-10698 Specify complex JobIDs for Lustre
• is related to: LU-12330 Support per-session JOBID without using the environment
• is related to: LU-16766 Combine some kernel process names for jobid

Comments

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" merged in patch https://review.whamcloud.com/c/doc/manual/+/58598/
Subject: LU-17512 jobid: describe '%j?' jobid_name conditional
Project: doc/manual
Branch: master
Current Patch Set:
Commit: 4762b758fb12a35fc9c80090941a7b7b2c6a1326

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/doc/manual/+/58598
Subject: LU-17512 jobid: describe '%j?' jobid_name conditional
Project: doc/manual
Branch: master
Current Patch Set: 1
Commit: 373f11918bfc13f552340e09e920f0fbffe89bcf

[Peter Jones]

Merged for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55332/
Subject: LU-17512 utils: new ? operator for jobid_name
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ecdcaa398668291dcfad26c4e21745dbfee19bc4

[Gerrit Updater]

"Maximilian Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55332
Subject: LU-17512 utils: new ? operator for jobid_name
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 3ca46d97d655cc89d30b613753107b78ad4d7756

[Andreas Dilger]

The "jobid_name" parameter can contain any of the "%" formats as desired by the admin. The goal of this ticket is to have a "conditional" operator like "?" in the jobid_name string that means "use the first parameter if set, and if not then use the second parameter."

The expected use case would be to have a format string like "%j?%H" that means "use the jobid if available, but if not then use the short hostname", but this is just an example of how it could be used. The jobid value is not always available (eg. on nodes with interactive users instead of compute nodes running a job), and a common usage is "jobid_name=%j:%e:%u" (jobid:executable:user) but if the jobid is unavailable then it prints as ":ls:0" which is ugly and not very useful.

Being able to conditionally use the short hostname in this case for the interactive nodes not running a job would be useful, and is the goal of this ticket.

To implement this, in jobid_interpret_string() it should check and print the first parameter (eg. jobid) as it does today but if it is NULL or zero-length (l == 0) and the next charactesr are '?%' then the second parameter should be printed, otherwise 'c' should be advanced by 3 characters to skip the "?%x" characters for the next format.

[Max Dilger]

For some clarification, do you want to use jobid_set_current to use the short hostname if a jobid is not currently set? Or to just return the short host name in the expanded string.

[Andreas Dilger]

This also has the benefit of shortening the JobID string so that more useful information can fit into the current size limits.