Details

    • Technical task
    • Resolution: Fixed
    • Minor
    • Lustre 2.7.0
    • None
    • None
    • 14084

    Description

      Binaries and scripts used only for debugging purposes are now into the main Lustre package. It may confuse new users and waste of disk space. Also some scripts brings additional dependency from perl package. This can be critical for small or embedded systems where only Lustre client is going to be installed and perl is not a part of default installation.

      The proposal is to split main package into at least two different packages. The first package should contain minimal set of tools really required for Lustre support. The second one can contain the rest of binaries and scripts.

      Attachments

        Issue Links

          Activity

            [LU-5104] Split binaries across packages more smart

            I wrongly suspected this ticket to cause some incmpatibilities, resolving again, sorry.

            bfaccini Bruno Faccini (Inactive) added a comment - I wrongly suspected this ticket to cause some incmpatibilities, resolving again, sorry.

            Gerrit change #10654 for this ticket breaks LU-1032/LU-5465 creation of Lustre Server DKMS RPMs, because now mount_osd_[zfs,ldiskfs].so lib is part of lustre-osd-[zfs,ldiskfs] binary RPM but is not part/built of/by DKMS RPM.
            We need to find a way to fix this, or this will lead to Lustre DKMS Server modules RPM to be unusable, at least for IEEL deployments ...

            bfaccini Bruno Faccini (Inactive) added a comment - Gerrit change #10654 for this ticket breaks LU-1032 / LU-5465 creation of Lustre Server DKMS RPMs, because now mount_osd_ [zfs,ldiskfs] .so lib is part of lustre-osd- [zfs,ldiskfs] binary RPM but is not part/built of/by DKMS RPM. We need to find a way to fix this, or this will lead to Lustre DKMS Server modules RPM to be unusable, at least for IEEL deployments ...

            Patch landed to master.

            dmiter Dmitry Eremin (Inactive) added a comment - Patch landed to master.

            Great. Thanks for comments. Is it Ok to have the following list of files in client package?

            Many of those commands should be installed on both client and servers nodes, e.g. lfs, lctl. At the very lest, these should only be installed as part of a server specific package:

            • /usr/sbin/ll_decode_filter_fid
            • /usr/sbin/lshowmount (although this is likely to be removed altogether under http://review.whamcloud.com/10510/)
            • /etc/ldev.conf
            • /etc/udev/rules.d/99-lustre.rules
            • /usr/sbin/ldev
            morrone Christopher Morrone (Inactive) added a comment - Great. Thanks for comments. Is it Ok to have the following list of files in client package? Many of those commands should be installed on both client and servers nodes, e.g. lfs, lctl. At the very lest, these should only be installed as part of a server specific package: /usr/sbin/ll_decode_filter_fid /usr/sbin/lshowmount (although this is likely to be removed altogether under http://review.whamcloud.com/10510/ ) /etc/ldev.conf /etc/udev/rules.d/99-lustre.rules /usr/sbin/ldev

            There could certainly be some overlap, but they are mostly distinct issues. This ticket deals with moving binaries mostly among the existing sub-packages. LU-3947 would create a new sub-package altogether. To my knowledge, no one is currently working on LU-3947, so the risk of conflicts is low.

            morrone Christopher Morrone (Inactive) added a comment - There could certainly be some overlap, but they are mostly distinct issues. This ticket deals with moving binaries mostly among the existing sub-packages. LU-3947 would create a new sub-package altogether. To my knowledge, no one is currently working on LU-3947 , so the risk of conflicts is low.

            Thanks. Yes, it overlaps.

            dmiter Dmitry Eremin (Inactive) added a comment - Thanks. Yes, it overlaps.

            John Hammond just submitted a patch; http://review.whamcloud.com/#/c/10510; under LU-2675 that removes a bunch of unused scripts. Looks like this also over laps this work.

            simmonsja James A Simmons added a comment - John Hammond just submitted a patch; http://review.whamcloud.com/#/c/10510 ; under LU-2675 that removes a bunch of unused scripts. Looks like this also over laps this work.

            The ll_decide_filter_fid and llog_reader tools are also only useful for the server. I think routerstat is useful in clients also, but not sure.

            As for llstat and llobdstat, I think that those used to be written in bash a long time ago, but it would need some investigation in old versions of Lustre.

            adilger Andreas Dilger added a comment - The ll_decide_filter_fid and llog_reader tools are also only useful for the server. I think routerstat is useful in clients also, but not sure. As for llstat and llobdstat, I think that those used to be written in bash a long time ago, but it would need some investigation in old versions of Lustre.

            This work is looking like it is overlapping with LU-3957

            simmonsja James A Simmons added a comment - This work is looking like it is overlapping with LU-3957

            Great. Thanks for comments. Is it Ok to have the following list of files in client package?

            ./sbin/mount.lustre
            ./usr/sbin/lst
            ./usr/sbin/ll_decode_filter_fid
            ./usr/sbin/lctl
            ./usr/sbin/lustre_rmmod
            ./usr/sbin/lshowmount
            ./usr/sbin/lhsmtool_posix
            ./usr/sbin/ptlctl
            ./usr/sbin/debugctl
            ./usr/bin/lfs
            ./usr/bin/lfs_migrate
            ./etc/init.d/lnet
            ./etc/ldev.conf
            ./etc/udev/rules.d/99-lustre.rules
            

            After rearrangement I have the following unspecified binaries:

            ./usr/sbin/llog_reader
            ./usr/sbin/routerstat
            ./usr/sbin/ldev           <- perl
            ./usr/sbin/ltrack_stats
            ./usr/bin/llstat          <- perl
            ./usr/bin/plot-llstat     <- perl
            ./usr/bin/llobdstat       <- perl
            
            dmiter Dmitry Eremin (Inactive) added a comment - Great. Thanks for comments. Is it Ok to have the following list of files in client package? ./sbin/mount.lustre ./usr/sbin/lst ./usr/sbin/ll_decode_filter_fid ./usr/sbin/lctl ./usr/sbin/lustre_rmmod ./usr/sbin/lshowmount ./usr/sbin/lhsmtool_posix ./usr/sbin/ptlctl ./usr/sbin/debugctl ./usr/bin/lfs ./usr/bin/lfs_migrate ./etc/init.d/lnet ./etc/ldev.conf ./etc/udev/rules.d/99-lustre.rules After rearrangement I have the following unspecified binaries: ./usr/sbin/llog_reader ./usr/sbin/routerstat ./usr/sbin/ldev <- perl ./usr/sbin/ltrack_stats ./usr/bin/llstat <- perl ./usr/bin/plot-llstat <- perl ./usr/bin/llobdstat <- perl

            I think some of these commands could just be removed:

            • llbackup - I don't think anyone is using that
            • lrun - that is for liblustre
            • loadgen - that could move into lustre-tests or lustre-iokit
            • lustre_config, lustre_createcsv - I don't think anyone uses these
            • lustre_up14 - ancient tool to upgrade 1.4 filesystems to 1.6
            • req_layout - could move into lustre-tests, or not be installed at all since it is really a developer tool

            Only needed on the server:

            • l_getidentity
            • llverdev
            • llverfs
            • ll_recover_lost_found_objs
            • lr_reader
            • lustre_req_history
            • lc_cluman
            • lc_hb
            • lc_lvm
            • lc_md
            • lc_modprobe
            • lc_net
            • lc_servip
            • lhbadm

            The files in lustre-iokit are already being renamed in http://review.whamcloud.com/10183 so that they have better names. I don't think they are relevant to this bug, since they are in a separate optional package already.

            I'm not sure why "lstats.sh" (which is renamed to "iokit-lstats" in my patch) should be moved out of lustre-iokit? It is a shell script anyway.

            I think that addresses most of the issues you have. There are a few Perl scripts left that are still useful in some cases, but you could just exclude those from the Xeon Phi packages if you think they are more trouble than they are worth.

            adilger Andreas Dilger added a comment - I think some of these commands could just be removed: llbackup - I don't think anyone is using that lrun - that is for liblustre loadgen - that could move into lustre-tests or lustre-iokit lustre_config, lustre_createcsv - I don't think anyone uses these lustre_up14 - ancient tool to upgrade 1.4 filesystems to 1.6 req_layout - could move into lustre-tests, or not be installed at all since it is really a developer tool Only needed on the server: l_getidentity llverdev llverfs ll_recover_lost_found_objs lr_reader lustre_req_history lc_cluman lc_hb lc_lvm lc_md lc_modprobe lc_net lc_servip lhbadm The files in lustre-iokit are already being renamed in http://review.whamcloud.com/10183 so that they have better names. I don't think they are relevant to this bug, since they are in a separate optional package already. I'm not sure why "lstats.sh" (which is renamed to "iokit-lstats" in my patch) should be moved out of lustre-iokit? It is a shell script anyway. I think that addresses most of the issues you have. There are a few Perl scripts left that are still useful in some cases, but you could just exclude those from the Xeon Phi packages if you think they are more trouble than they are worth.

            People

              dmiter Dmitry Eremin (Inactive)
              dmiter Dmitry Eremin (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: