[LU-10378] "lfs find" is missing "-printf" support Created: 13/Dec/17  Updated: 15/Feb/23  Resolved: 26/Jan/22

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.10.0
Fix Version/s: Lustre 2.15.0

Type: Improvement Priority: Minor
Reporter: Andreas Dilger Assignee: Anjus George
Resolution: Fixed Votes: 0
Labels: medium

Issue Links:
Cloners
is cloned by LU-15504 "lfs find" is missing "-ls" support Open
Duplicate
is duplicated by LU-9689 lfs getstripe does not print globbed ... Resolved
Related
is related to LU-5170 lfs usability Open
is related to LU-7495 lfs find is missing "-links" support Resolved
is related to LU-16560 'lfs find -printf %w' does not print ... Open
is related to LU-15837 "lfs find -printf" improvements Open
is related to LU-15276 DNE3: "lfs find" able to check defaul... Open
is related to LU-9537 implement "lfs getstripe --fid" for d... Resolved
Rank (Obsolete): 9223372036854775807

 Description   

It would be useful to be able to add the "-ls" option to "lfs find" to print out file attributes.

Even better would be if -ls was implemented as a fixed-format option of the "--printf" command, which could print out POSIX and Lustre-specific attributes such as OST/MDT indices for stripes, MDT inode index, etc. similar to find(1) and stat(1).

Unfortunately, the % options used by find(1) and stat(1) are not consistent. In case of disagreement, the find(1) options should be used. Thankfully, it appears that "%L" is not used for --printf by either find(1) or stat(1), so it could be used to print out the Lustre-specific options, for example "%Li" for starting OST index, "%Lo" for all OST indices, "%Lc" for stripe count, "%LS" for stripe size, "%Lp" for pool, etc., with the second letter matching the short options used by lfs getstripe.

It would also be useful to add --printf to "lfs getstripe" as well.



 Comments   
Comment by Rick Mohr [ 14/May/21 ]

Myself and a colleague are working on implementing a feature to "lfs find" that would print additional information about files beyond just the filename.  Initially, we are looking to add a new option (maybe something like --print-all) that would print most (if not all) information about matching files, like stripe count, layout info, timestamps, etc. in some form of structured output suitable for parsing by other programs.  The plan is to use this as a foundation for later adding a --printf option that would allow users to specify what subset of information they want and the format they want it in.

Would Whamcloud be open to accepting a feature like --print-all with the expectation that future patches would implement the --printf option?

Comment by Andreas Dilger [ 18/May/21 ]

I'm reluctant to give a "yes, I will accept anything" approval in advance of seeing the patch, or at least some inclination what the "structured output" would look like. However, I'm be generally in favor of something like this, e.g. "--print-yaml" that used an extensible basic YAML format. I'd be less keen on something like "--print-csv" or colon-separated values that are positional, because adding new fields in the future would be difficult.

Given that we already have "lfs getstripe --yaml" for printing the trusted.lov (lustre.lov) xattr, and that might be considered part of "all" inode attributes, then it makes sense to be able encapsulate that (and other xattrs) into the output in the future, rather than print all of the fields separately (and which only make sense for plain layouts).

Comment by Rick Mohr [ 18/May/21 ]

I wasn't necessarily looking for an unconditional "yes", but I just wanted to make sure it wasn't going to be an unconditional "no".

YAML is one format we are considering since that would be consistent with "lfs getstripe --yaml" as you mentioned.  We had considered just trying to convert the structures to yaml format to print them out, but now we are thinking that might print out more info than we actually need.   Plus, some info (like project id) does not appear to be contained in structures like lov_user_mds_data anyway.

So we're looking into compiling a list of the values that we might be interested in printing using a printf syntax and also creating an initial list of the format specifiers (like %Lo) that would go with each one (to prepare for eventual printf implementation).    Then perhaps the --print-all option could just be a shortcut for something like --printf "stripe_count: %Lc \n stripe_size: %Ls \n ...".

(I am just using --print-all as a placeholder for whatever name we choose, not as a final design decision.  And I can absolutely assure you the format will not be CSV )

Comment by Gerrit Updater [ 15/Sep/21 ]

"Anjus George <georgea@ornl.gov>" uploaded a new patch: https://review.whamcloud.com/44929
Subject: LU-10378 utils: add formatted printf to lfs find
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 46e2503acbf86649f0e88c7b13b012677e5aac3b

Comment by Gerrit Updater [ 06/Oct/21 ]

"Anjus George <georgea@ornl.gov>" uploaded a new patch: https://review.whamcloud.com/45136
Subject: LU-10378 utils: add formatted printf to lfs find
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8c6526f39abfcb2625c51f9cf098b8c73a2e0dda

Comment by Andreas Dilger [ 24/Nov/21 ]

Assign to Anjus who is working on a patch for "lfs find -printf".

Comment by Gerrit Updater [ 26/Jan/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45136/
Subject: LU-10378 utils: add formatted printf to lfs find
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 6b8e97b76c472068e7d6bc792e4f202b2f70ca67

Comment by Peter Jones [ 26/Jan/22 ]

Landed for 2.15

Comment by Andreas Dilger [ 31/Jan/22 ]

The landed patch added "-printf" support, but not "-ls". I'll clone this to add a separate LU ticket for "-ls".

Generated at Sat Feb 10 02:34:31 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.