Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10378

"lfs find" is missing "-printf" support

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • Lustre 2.15.0
    • Lustre 2.10.0
    • 9223372036854775807

    Description

      It would be useful to be able to add the "-ls" option to "lfs find" to print out file attributes.

      Even better would be if -ls was implemented as a fixed-format option of the "--printf" command, which could print out POSIX and Lustre-specific attributes such as OST/MDT indices for stripes, MDT inode index, etc. similar to find(1) and stat(1).

      Unfortunately, the % options used by find(1) and stat(1) are not consistent. In case of disagreement, the find(1) options should be used. Thankfully, it appears that "%L" is not used for --printf by either find(1) or stat(1), so it could be used to print out the Lustre-specific options, for example "%Li" for starting OST index, "%Lo" for all OST indices, "%Lc" for stripe count, "%LS" for stripe size, "%Lp" for pool, etc., with the second letter matching the short options used by lfs getstripe.

      It would also be useful to add --printf to "lfs getstripe" as well.

      Attachments

        Issue Links

          Activity

            [LU-10378] "lfs find" is missing "-printf" support

            The landed patch added "-printf" support, but not "-ls". I'll clone this to add a separate LU ticket for "-ls".

            adilger Andreas Dilger added a comment - The landed patch added " -printf " support, but not " -ls ". I'll clone this to add a separate LU ticket for " -ls ".
            pjones Peter Jones added a comment -

            Landed for 2.15

            pjones Peter Jones added a comment - Landed for 2.15

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45136/
            Subject: LU-10378 utils: add formatted printf to lfs find
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 6b8e97b76c472068e7d6bc792e4f202b2f70ca67

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45136/ Subject: LU-10378 utils: add formatted printf to lfs find Project: fs/lustre-release Branch: master Current Patch Set: Commit: 6b8e97b76c472068e7d6bc792e4f202b2f70ca67

            Assign to Anjus who is working on a patch for "lfs find -printf".

            adilger Andreas Dilger added a comment - Assign to Anjus who is working on a patch for " lfs find -printf ".

            "Anjus George <georgea@ornl.gov>" uploaded a new patch: https://review.whamcloud.com/45136
            Subject: LU-10378 utils: add formatted printf to lfs find
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 8c6526f39abfcb2625c51f9cf098b8c73a2e0dda

            gerrit Gerrit Updater added a comment - "Anjus George <georgea@ornl.gov>" uploaded a new patch: https://review.whamcloud.com/45136 Subject: LU-10378 utils: add formatted printf to lfs find Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 8c6526f39abfcb2625c51f9cf098b8c73a2e0dda
            gerrit Gerrit Updater added a comment - - edited

            "Anjus George <georgea@ornl.gov>" uploaded a new patch: https://review.whamcloud.com/44929
            Subject: LU-10378 utils: add formatted printf to lfs find
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 46e2503acbf86649f0e88c7b13b012677e5aac3b

            gerrit Gerrit Updater added a comment - - edited "Anjus George <georgea@ornl.gov>" uploaded a new patch: https://review.whamcloud.com/44929 Subject: LU-10378 utils: add formatted printf to lfs find Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 46e2503acbf86649f0e88c7b13b012677e5aac3b
            rmohr Rick Mohr added a comment -

            I wasn't necessarily looking for an unconditional "yes", but I just wanted to make sure it wasn't going to be an unconditional "no".

            YAML is one format we are considering since that would be consistent with "lfs getstripe --yaml" as you mentioned.  We had considered just trying to convert the structures to yaml format to print them out, but now we are thinking that might print out more info than we actually need.   Plus, some info (like project id) does not appear to be contained in structures like lov_user_mds_data anyway.

            So we're looking into compiling a list of the values that we might be interested in printing using a printf syntax and also creating an initial list of the format specifiers (like %Lo) that would go with each one (to prepare for eventual printf implementation).    Then perhaps the --print-all option could just be a shortcut for something like --printf "stripe_count: %Lc \n stripe_size: %Ls \n ...".

            (I am just using --print-all as a placeholder for whatever name we choose, not as a final design decision.  And I can absolutely assure you the format will not be CSV )

            rmohr Rick Mohr added a comment - I wasn't necessarily looking for an unconditional "yes", but I just wanted to make sure it wasn't going to be an unconditional "no". YAML is one format we are considering since that would be consistent with " lfs getstripe --yaml " as you mentioned.  We had considered just trying to convert the structures to yaml format to print them out, but now we are thinking that might print out more info than we actually need.   Plus, some info (like project id) does not appear to be contained in structures like lov_user_mds_data anyway. So we're looking into compiling a list of the values that we might be interested in printing using a printf syntax and also creating an initial list of the format specifiers (like %Lo) that would go with each one (to prepare for eventual printf implementation).    Then perhaps the --print-all option could just be a shortcut for something like --printf "stripe_count: %Lc \n stripe_size: %Ls \n ...". (I am just using --print-all as a placeholder for whatever name we choose, not as a final design decision.  And I can absolutely assure you the format will not be CSV )
            adilger Andreas Dilger added a comment - - edited

            I'm reluctant to give a "yes, I will accept anything" approval in advance of seeing the patch, or at least some inclination what the "structured output" would look like. However, I'm be generally in favor of something like this, e.g. "--print-yaml" that used an extensible basic YAML format. I'd be less keen on something like "--print-csv" or colon-separated values that are positional, because adding new fields in the future would be difficult.

            Given that we already have "lfs getstripe --yaml" for printing the trusted.lov (lustre.lov) xattr, and that might be considered part of "all" inode attributes, then it makes sense to be able encapsulate that (and other xattrs) into the output in the future, rather than print all of the fields separately (and which only make sense for plain layouts).

            adilger Andreas Dilger added a comment - - edited I'm reluctant to give a "yes, I will accept anything" approval in advance of seeing the patch, or at least some inclination what the "structured output" would look like. However, I'm be generally in favor of something like this, e.g. " --print-yaml " that used an extensible basic YAML format. I'd be less keen on something like " --print-csv " or colon-separated values that are positional, because adding new fields in the future would be difficult. Given that we already have " lfs getstripe --yaml " for printing the trusted.lov ( lustre.lov ) xattr, and that might be considered part of "all" inode attributes, then it makes sense to be able encapsulate that (and other xattrs) into the output in the future, rather than print all of the fields separately (and which only make sense for plain layouts).
            rmohr Rick Mohr added a comment -

            Myself and a colleague are working on implementing a feature to "lfs find" that would print additional information about files beyond just the filename.  Initially, we are looking to add a new option (maybe something like --print-all) that would print most (if not all) information about matching files, like stripe count, layout info, timestamps, etc. in some form of structured output suitable for parsing by other programs.  The plan is to use this as a foundation for later adding a --printf option that would allow users to specify what subset of information they want and the format they want it in.

            Would Whamcloud be open to accepting a feature like --print-all with the expectation that future patches would implement the --printf option?

            rmohr Rick Mohr added a comment - Myself and a colleague are working on implementing a feature to "lfs find" that would print additional information about files beyond just the filename.  Initially, we are looking to add a new option (maybe something like --print-all) that would print most (if not all) information about matching files, like stripe count, layout info, timestamps, etc. in some form of structured output suitable for parsing by other programs.  The plan is to use this as a foundation for later adding a --printf option that would allow users to specify what subset of information they want and the format they want it in. Would Whamcloud be open to accepting a feature like --print-all with the expectation that future patches would implement the --printf option?

            People

              georgea Anjus George
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: