Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-15504

"lfs find" is missing "-ls" support

Details

    • 9223372036854775807

    Description

      It would be useful to be able to add the "-ls" option to "lfs find" to print out file attributes.

      Even better would be if -ls was implemented as a fixed-format option of the "--printf" command, which could print out POSIX and possibly some limited Lustre-specific attributes such as FID, stripe_count.

      Attachments

        Issue Links

          Clones

          Improvement - An improvement or enhancement to an existing feature or task. LU-10378 "lfs find" is missing "-printf" support

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. LU-9689 lfs getstripe does not print globbed file names when using options such as --stripe-count and --stripe-size

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-16560 'lfs find -printf %w' does not print birth time

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-15276 DNE3: "lfs find" able to check default directory and file layout

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-18889 "lfs find -printf" optimization

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-18890 add "lfs find -ls" option to lfs-find.1 man page

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-15837 "lfs find -printf" improvements

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-9537 implement "lfs getstripe --fid" for directory FIDs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-16561 allow 'lfs find -printf' field widths

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-5170 lfs usability

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-7495 lfs find is missing "-links" support

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-15504] "lfs find" is missing "-ls" support
            pjones Peter Jones added a comment -

            Landed for 2.16

            pjones Peter Jones added a comment - Landed for 2.16
            adilger Andreas Dilger added a comment -

            @Max, please update the lustre/doc/lfs-find.1 man page to describe the new "ls" functionality.

            adilger Andreas Dilger added a comment - @Max, please update the lustre/doc/lfs-find.1 man page to describe the new " ls " functionality.
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55443/
            Subject: LU-15504 utils: lfs find -ls function
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 1d8164fa16efb12d02b0855de61eb95d7ba246ab

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55443/ Subject: LU-15504 utils: lfs find -ls function Project: fs/lustre-release Branch: master Current Patch Set: Commit: 1d8164fa16efb12d02b0855de61eb95d7ba246ab
            gerrit Gerrit Updater added a comment -

            "Maximilian Dilger <mdilger@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55443
            Subject: LU-15504 utils: lfs find -ls function
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 647b29ac27691afce21775499ea3671cc565c08c

            gerrit Gerrit Updater added a comment - "Maximilian Dilger <mdilger@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55443 Subject: LU-15504 utils: lfs find -ls function Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 647b29ac27691afce21775499ea3671cc565c08c
            adilger Andreas Dilger added a comment -

            I also filed LU-16561 to implement fixed field widths for "lfs find -printf", which would probably give better results for "lfs find -ls" than using "\t" for the field separator.

            adilger Andreas Dilger added a comment - I also filed LU-16561 to implement fixed field widths for " lfs find -printf ", which would probably give better results for " lfs find -ls " than using "\t" for the field separator.
            georgea Anjus George added a comment -

            @Andreas Dilger, Thank you for the detailed notes. Let me read through it and discuss with Rick.

            georgea Anjus George added a comment - @Andreas Dilger, Thank you for the detailed notes. Let me read through it and discuss with Rick.
            adilger Andreas Dilger added a comment - - edited

            georgea, is this something that you and/or rmohr are planning to implement for 2.16.0? If not, hopefully the following notes will be useful to you or whomever looks into this in the future.

            I haven't looked into the details, but it looks reasonable to have the "-ls" option internally map to "-printf" with a static format string to match the output format of the normal "find" command:

            $ find /etc -ls | head -10
130563   12 drwxr-xr-x  94   root   root      12288 Oct 25 17:15 /etc
146902    4 drwxr-xr-x   3   root   root       4096 Sep 30 13:41 /etc/tuned
131968    4 -rw-r--r--   1   root   root       1305 Sep 21 07:17 /etc/tuned/tuned-main.conf
131975    4 -rw-r--r--   1   root   root         14 Oct 21 14:12 /etc/tuned/active_profile
146903    4 drwxr-xr-x   2   root   root       4096 Sep 21 07:17 /etc/tuned/recommend.d
146040    4 -rw-r--r--   1   root   root          5 Oct 21 14:12 /etc/tuned/profile_mode
131967    4 -rw-r--r--   1   root   root       1111 Mar 21  2019 /etc/tuned/bootcmdline
130615    4 ----------   1   root   root        859 Jul 20  2020 /etc/shadow-
130967    0 lrwxrwxrwx   1   root   root         10 Jan 25  2022 /etc/rc0.d -> rc.d/rc0.d
167146    4 -rw-r--r--   1   root   root       1968 Apr 11  2018 /etc/mail.rc
#inode  kbs   sym_mode lnk usrnam grpnam size_bytes  modify_time pathname

            This looks like "%i\t%k\t%M\t%n\t%u\t%g\t%s\t%t\t%p\n" would be at least a good first approximation, according to the Linux manpage.

            It looks like a few additional -printf options would need to be implemented, but nothing overly complex:

            • "%i" - inode number (should be trivial).  For Lustre it might be more useful to print "%LF" for the FID instead of the inode number"?
            • "%k" - file blocks in kilobytes (should be trivial, just "%b/2")
            • "%M" - symbolic file access mode (a small amount of work, decode file type (-bcdls) + -rwxSt UGO permission bits)
            • "%n" - link count (should be trivial)
            • "%u" - username (could be mapped to "%U" initially to print numeric UID, and implemented separately)
            • "%g" - groupname (use "%g" initially, as with "%u")

            Comparing the output of regular "find -printf '%i\t%k\t%M\t%n\t%u\t%g\t%s\t%t\t%p\n'" with "find -ls" shows they are fairly close:

            [root@centos7 ~]# find /etc -printf "%i\t%k\t%M\t%n\t%u\t%g\t%s\t%t\t%p\n" | head -10
130563	12	drwxr-xr-x	94	root	root	12288	Tue Oct 25 17:15:22.0799592647 2022	/etc
146902	4	drwxr-xr-x	3	root	root	4096	Fri Sep 30 13:41:40.0933994189 2022	/etc/tuned
131968	4	-rw-r--r--	1	root	root	1305	Wed Sep 21 07:17:58.0000000000 2022	/etc/tuned/tuned-main.conf
131975	4	-rw-r--r--	1	root	root	14	Fri Oct 21 14:12:37.0195947071 2022	/etc/tuned/active_profile
146903	4	drwxr-xr-x	2	root	root	4096	Wed Sep 21 07:17:58.0000000000 2022	/etc/tuned/recommend.d
146040	4	-rw-r--r--	1	root	root	5	Fri Oct 21 14:12:37.0198948473 2022	/etc/tuned/profile_mode
131967	4	-rw-r--r--	1	root	root	1111	Thu Mar 21 16:10:46.0000000000 2019	/etc/tuned/bootcmdline
130615	4	----------	1	root	root	859	Mon Jul 20 18:27:36.0000000000 2020	/etc/shadow-
130861	0	lrwxrwxrwx	1	root	root	13	Fri Sep 30 13:41:15.0672852655 2022	/etc/rc.local
130847	4	drwxr-xr-x	2	root	root	4096	Fri Sep 30 13:41:39.0487986087 2022	/etc/modprobe.d
[root@centos7 ~]# find /etc -ls | head -10
130563   12 drwxr-xr-x  94 root     root        12288 Oct 25 17:15 /etc
146902    4 drwxr-xr-x   3 root     root         4096 Sep 30 13:41 /etc/tuned
131968    4 -rw-r--r--   1 root     root         1305 Sep 21 07:17 /etc/tuned/tuned-main.conf
131975    4 -rw-r--r--   1 root     root           14 Oct 21 14:12 /etc/tuned/active_profile
146903    4 drwxr-xr-x   2 root     root         4096 Sep 21 07:17 /etc/tuned/recommend.d
146040    4 -rw-r--r--   1 root     root            5 Oct 21 14:12 /etc/tuned/profile_mode
131967    4 -rw-r--r--   1 root     root         1111 Mar 21  2019 /etc/tuned/bootcmdline
130615    4 ----------   1 root     root          859 Jul 20  2020 /etc/shadow-
130861    0 lrwxrwxrwx   1 root     root           13 Sep 30 13:41 /etc/rc.local -> rc.d/rc.local
130847    4 drwxr-xr-x   2 root     root         4096 Sep 30 13:41 /etc/modprobe.d

            Printing out the symlink target would also need a small amount of extra work, not sure how critical that would be for an initial implementation. The timestamps are abbreviated to 3 columns ("Mon Day Time", or "Mon Day Year" for files over 6 months old), but I'm not sure that is better than having a fixed-format timestamp (though it is a bit longer). Some of the column widths are smaller than a whole tab also, and right justified for the numbers, but I don't think that is totally critical.

            adilger Andreas Dilger added a comment - - edited georgea , is this something that you and/or rmohr are planning to implement for 2.16.0? If not, hopefully the following notes will be useful to you or whomever looks into this in the future. I haven't looked into the details, but it looks reasonable to have the " -ls " option internally map to " -printf " with a static format string to match the output format of the normal " find " command: $ find /etc -ls | head -10 130563 12 drwxr-xr-x 94 root root 12288 Oct 25 17:15 /etc 146902 4 drwxr-xr-x 3 root root 4096 Sep 30 13:41 /etc/tuned 131968 4 -rw-r--r-- 1 root root 1305 Sep 21 07:17 /etc/tuned/tuned-main.conf 131975 4 -rw-r--r-- 1 root root 14 Oct 21 14:12 /etc/tuned/active_profile 146903 4 drwxr-xr-x 2 root root 4096 Sep 21 07:17 /etc/tuned/recommend.d 146040 4 -rw-r--r-- 1 root root 5 Oct 21 14:12 /etc/tuned/profile_mode 131967 4 -rw-r--r-- 1 root root 1111 Mar 21 2019 /etc/tuned/bootcmdline 130615 4 ---------- 1 root root 859 Jul 20 2020 /etc/shadow- 130967 0 lrwxrwxrwx 1 root root 10 Jan 25 2022 /etc/rc0.d -> rc.d/rc0.d 167146    4 -rw-r--r--   1 root   root       1968 Apr 11  2018 /etc/mail.rc #inode kbs sym_mode lnk usrnam grpnam size_bytes modify_time pathname This looks like " %i\t%k\t%M\t%n\t%u\t%g\t%s\t%t\t%p \n" would be at least a good first approximation, according to the Linux manpage. It looks like a few additional -printf options would need to be implemented, but nothing overly complex: " %i " - inode number (should be trivial).  For Lustre it might be more useful to print " %LF " for the FID instead of the inode number"? " %k " - file blocks in kilobytes (should be trivial, just " %b/2 ") " %M " - symbolic file access mode (a small amount of work, decode file type ( -bcdls ) + -rwxSt UGO permission bits) " %n " - link count (should be trivial) " %u " - username (could be mapped to " %U " initially to print numeric UID, and implemented separately) " %g " - groupname (use " %g " initially, as with " %u ") Comparing the output of regular " find -printf '%i\t%k\t%M\t%n\t%u\t%g\t%s\t%t\t%p \n'" with " find -ls " shows they are fairly close: [root@centos7 ~]# find /etc -printf "%i\t%k\t%M\t%n\t%u\t%g\t%s\t%t\t%p\n" | head -10 130563 12 drwxr-xr-x 94 root root 12288 Tue Oct 25 17:15:22.0799592647 2022 /etc 146902 4 drwxr-xr-x 3 root root 4096 Fri Sep 30 13:41:40.0933994189 2022 /etc/tuned 131968 4 -rw-r--r-- 1 root root 1305 Wed Sep 21 07:17:58.0000000000 2022 /etc/tuned/tuned-main.conf 131975 4 -rw-r--r-- 1 root root 14 Fri Oct 21 14:12:37.0195947071 2022 /etc/tuned/active_profile 146903 4 drwxr-xr-x 2 root root 4096 Wed Sep 21 07:17:58.0000000000 2022 /etc/tuned/recommend.d 146040 4 -rw-r--r-- 1 root root 5 Fri Oct 21 14:12:37.0198948473 2022 /etc/tuned/profile_mode 131967 4 -rw-r--r-- 1 root root 1111 Thu Mar 21 16:10:46.0000000000 2019 /etc/tuned/bootcmdline 130615 4 ---------- 1 root root 859 Mon Jul 20 18:27:36.0000000000 2020 /etc/shadow- 130861 0 lrwxrwxrwx 1 root root 13 Fri Sep 30 13:41:15.0672852655 2022 /etc/rc.local 130847 4 drwxr-xr-x 2 root root 4096 Fri Sep 30 13:41:39.0487986087 2022 /etc/modprobe.d [root@centos7 ~]# find /etc -ls | head -10 130563 12 drwxr-xr-x 94 root root 12288 Oct 25 17:15 /etc 146902 4 drwxr-xr-x 3 root root 4096 Sep 30 13:41 /etc/tuned 131968 4 -rw-r--r-- 1 root root 1305 Sep 21 07:17 /etc/tuned/tuned-main.conf 131975 4 -rw-r--r-- 1 root root 14 Oct 21 14:12 /etc/tuned/active_profile 146903 4 drwxr-xr-x 2 root root 4096 Sep 21 07:17 /etc/tuned/recommend.d 146040 4 -rw-r--r-- 1 root root 5 Oct 21 14:12 /etc/tuned/profile_mode 131967 4 -rw-r--r-- 1 root root 1111 Mar 21 2019 /etc/tuned/bootcmdline 130615 4 ---------- 1 root root 859 Jul 20 2020 /etc/shadow- 130861 0 lrwxrwxrwx 1 root root 13 Sep 30 13:41 /etc/rc.local -> rc.d/rc.local 130847 4 drwxr-xr-x 2 root root 4096 Sep 30 13:41 /etc/modprobe.d Printing out the symlink target would also need a small amount of extra work, not sure how critical that would be for an initial implementation. The timestamps are abbreviated to 3 columns (" Mon Day Time ", or " Mon Day Year " for files over 6 months old), but I'm not sure that is better than having a fixed-format timestamp (though it is a bit longer). Some of the column widths are smaller than a whole tab also, and right justified for the numbers, but I don't think that is totally critical.

            People

              mdilger Max Dilger
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: