[LU-5104] Split binaries across packages more smart

• Status: Closed
• Project: Lustre
• Fix Version/s: Lustre 2.7.0
• Type: Technical task
• Priority: Minor
• Reporter: Dmitry Eremin (Inactive)
• Assignee: Dmitry Eremin (Inactive)
• Resolution: Fixed

Description

Binaries and scripts used only for debugging purposes are now into the main Lustre package. It may confuse new users and waste of disk space. Also some scripts brings additional dependency from perl package. This can be critical for small or embedded systems where only Lustre client is going to be installed and perl is not a part of default installation.

The proposal is to split main package into at least two different packages. The first package should contain minimal set of tools really required for Lustre support. The second one can contain the rest of binaries and scripts.

Issue Links

• is related to: LU-3957 Create separate server and client bin...

Comments

[Andreas Dilger]

Could you please be specific about which binaries/scripts should be moved?

[Dmitry Eremin (Inactive)]

I propose that a main package should have only:
/usr/bin:

lfs
lfs_migrate
llbackup

/usr/sbin:

debugctl
l_getidentity
lc_cluman
lc_hb
lc_lvm
lc_md
lc_modprobe
lc_net
lc_servip
lctl
lhbadm
lhsmtool_posix
ll_decode_filter_fid
llog_reader
ll_recover_lost_found_objs
llverdev
llverfs
lshowmount
lst
lstclient
lustre_config
lustre_createcsv
lustre_rmmod
lustre_routes_config
lustre_routes_conversion
lustre_rsync
lustre_start
lustre_up14
ptlctl

lustre-iokit:
/usr/bin:

config.sh
lstats.sh
gather_stats_everywhere.sh
ior-survey
libecho
mds-survey
obdfilter-survey
ost-survey
parse-ior
plot-obdfilter
plot-ost
plot-sgpdd
sgpdd-survey

The additional package with utils will have a rest:
/usr/bin:

llstat
llobdstat
lustre_req_history
plot-llstat
req_layout

/usr/sbin:

ldev
loadgen
lr_reader
lrun
ltrack_stats
routerstat

[Andreas Dilger]

I think some of these commands could just be removed:

• llbackup - I don't think anyone is using that
• lrun - that is for liblustre
• loadgen - that could move into lustre-tests or lustre-iokit
• lustre_config, lustre_createcsv - I don't think anyone uses these
• lustre_up14 - ancient tool to upgrade 1.4 filesystems to 1.6
• req_layout - could move into lustre-tests, or not be installed at all since it is really a developer tool

Only needed on the server:

• l_getidentity
• llverdev
• llverfs
• ll_recover_lost_found_objs
• lr_reader
• lustre_req_history
• lc_cluman
• lc_hb
• lc_lvm
• lc_md
• lc_modprobe
• lc_net
• lc_servip
• lhbadm

The files in lustre-iokit are already being renamed in http://review.whamcloud.com/10183 so that they have better names. I don't think they are relevant to this bug, since they are in a separate optional package already.

I'm not sure why "lstats.sh" (which is renamed to "iokit-lstats" in my patch) should be moved out of lustre-iokit? It is a shell script anyway.

I think that addresses most of the issues you have. There are a few Perl scripts left that are still useful in some cases, but you could just exclude those from the Xeon Phi packages if you think they are more trouble than they are worth.

[Dmitry Eremin (Inactive)]

Great. Thanks for comments. Is it Ok to have the following list of files in client package?

./sbin/mount.lustre
./usr/sbin/lst
./usr/sbin/ll_decode_filter_fid
./usr/sbin/lctl
./usr/sbin/lustre_rmmod
./usr/sbin/lshowmount
./usr/sbin/lhsmtool_posix
./usr/sbin/ptlctl
./usr/sbin/debugctl
./usr/bin/lfs
./usr/bin/lfs_migrate
./etc/init.d/lnet
./etc/ldev.conf
./etc/udev/rules.d/99-lustre.rules

After rearrangement I have the following unspecified binaries:

./usr/sbin/llog_reader
./usr/sbin/routerstat
./usr/sbin/ldev           <- perl
./usr/sbin/ltrack_stats
./usr/bin/llstat          <- perl
./usr/bin/plot-llstat     <- perl
./usr/bin/llobdstat       <- perl

[James A Simmons]

This work is looking like it is overlapping with LU-3957

[Andreas Dilger]

The ll_decide_filter_fid and llog_reader tools are also only useful for the server. I think routerstat is useful in clients also, but not sure.

As for llstat and llobdstat, I think that those used to be written in bash a long time ago, but it would need some investigation in old versions of Lustre.

[James A Simmons]

John Hammond just submitted a patch; http://review.whamcloud.com/#/c/10510; under LU-2675 that removes a bunch of unused scripts. Looks like this also over laps this work.

[Dmitry Eremin (Inactive)]

Thanks. Yes, it overlaps.

[Christopher Morrone]

There could certainly be some overlap, but they are mostly distinct issues. This ticket deals with moving binaries mostly among the existing sub-packages. LU-3947 would create a new sub-package altogether. To my knowledge, no one is currently working on LU-3947, so the risk of conflicts is low.

[Christopher Morrone]

Great. Thanks for comments. Is it Ok to have the following list of files in client package?

Many of those commands should be installed on both client and servers nodes, e.g. lfs, lctl. At the very lest, these should only be installed as part of a server specific package:

• /usr/sbin/ll_decode_filter_fid
• /usr/sbin/lshowmount (although this is likely to be removed altogether under http://review.whamcloud.com/10510/)
• /etc/ldev.conf
• /etc/udev/rules.d/99-lustre.rules
• /usr/sbin/ldev

[Dmitry Eremin (Inactive)]

Patch landed to master.

[Bruno Faccini (Inactive)]

Gerrit change #10654 for this ticket breaks LU-1032/LU-5465 creation of Lustre Server DKMS RPMs, because now mount_osd_[zfs,ldiskfs].so lib is part of lustre-osd-[zfs,ldiskfs] binary RPM but is not part/built of/by DKMS RPM.
We need to find a way to fix this, or this will lead to Lustre DKMS Server modules RPM to be unusable, at least for IEEL deployments ...

[Bruno Faccini (Inactive)]

I wrongly suspected this ticket to cause some incmpatibilities, resolving again, sorry.