https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5104 Lustre <p>Binaries and scripts used only for debugging purposes are now into the main Lustre package. It may confuse new users and waste of disk space. Also some scripts brings additional dependency from perl package. This can be critical for small or embedded systems where only Lustre client is going to be installed and perl is not a part of default installation.</p> <p>The proposal is to split main package into at least two different packages. The first package should contain minimal set of tools really required for Lustre support. The second one can contain the rest of binaries and scripts.</p> LU-5104

Split binaries across packages more smart

Technical task LU-3953 Minor Closed Fixed Dmitry Eremin Dmitry Eremin Tue, 27 May 2014 12:26:15 +0000 Thu, 5 May 2016 21:12:37 +0000 Tue, 21 Oct 2014 11:33:06 +0000 Lustre 2.7.0 0 8 <p>Could you please be specific about which binaries/scripts should be moved?</p> <p>I propose that a main package should have only:<br/> /usr/bin:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lfs lfs_migrate llbackup </pre> </div></div> <p>/usr/sbin:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>debugctl l_getidentity lc_cluman lc_hb lc_lvm lc_md lc_modprobe lc_net lc_servip lctl lhbadm lhsmtool_posix ll_decode_filter_fid llog_reader ll_recover_lost_found_objs llverdev llverfs lshowmount lst lstclient lustre_config lustre_createcsv lustre_rmmod lustre_routes_config lustre_routes_conversion lustre_rsync lustre_start lustre_up14 ptlctl </pre> </div></div> <p>lustre-iokit:<br/> /usr/bin:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>config.sh lstats.sh gather_stats_everywhere.sh ior-survey libecho mds-survey obdfilter-survey ost-survey parse-ior plot-obdfilter plot-ost plot-sgpdd sgpdd-survey </pre> </div></div> <p>The additional package with utils will have a rest:<br/> /usr/bin:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>llstat llobdstat lustre_req_history plot-llstat req_layout </pre> </div></div> <p>/usr/sbin:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>ldev loadgen lr_reader lrun ltrack_stats routerstat </pre> </div></div> <p>I think some of these commands could just be removed:</p> <ul class="alternate" type="square"> <li>llbackup - I don't think anyone is using that</li> <li>lrun - that is for liblustre</li> <li>loadgen - that could move into lustre-tests or lustre-iokit</li> <li>lustre_config, lustre_createcsv - I don't think anyone uses these</li> <li>lustre_up14 - ancient tool to upgrade 1.4 filesystems to 1.6</li> <li>req_layout - could move into lustre-tests, or not be installed at all since it is really a developer tool</li> </ul> <p>Only needed on the server:</p> <ul class="alternate" type="square"> <li>l_getidentity</li> <li>llverdev</li> <li>llverfs</li> <li>ll_recover_lost_found_objs</li> <li>lr_reader</li> <li>lustre_req_history</li> <li>lc_cluman</li> <li>lc_hb</li> <li>lc_lvm</li> <li>lc_md</li> <li>lc_modprobe</li> <li>lc_net</li> <li>lc_servip</li> <li>lhbadm</li> </ul> <p>The files in lustre-iokit are already being renamed in <a href="http://review.whamcloud.com/10183" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10183</a> so that they have better names. I don't think they are relevant to this bug, since they are in a separate optional package already.</p> <p>I'm not sure why "lstats.sh" (which is renamed to "iokit-lstats" in my patch) should be moved out of lustre-iokit? It is a shell script anyway.</p> <p>I think that addresses most of the issues you have. There are a few Perl scripts left that are still useful in some cases, but you could just exclude those from the Xeon Phi packages if you think they are more trouble than they are worth.</p> <p>Great. Thanks for comments. Is it Ok to have the following list of files in <b>client</b> package?</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>./sbin/mount.lustre ./usr/sbin/lst ./usr/sbin/ll_decode_filter_fid ./usr/sbin/lctl ./usr/sbin/lustre_rmmod ./usr/sbin/lshowmount ./usr/sbin/lhsmtool_posix ./usr/sbin/ptlctl ./usr/sbin/debugctl ./usr/bin/lfs ./usr/bin/lfs_migrate ./etc/init.d/lnet ./etc/ldev.conf ./etc/udev/rules.d/99-lustre.rules </pre> </div></div> <p>After rearrangement I have the following unspecified binaries:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>./usr/sbin/llog_reader ./usr/sbin/routerstat ./usr/sbin/ldev &lt;- perl ./usr/sbin/ltrack_stats ./usr/bin/llstat &lt;- perl ./usr/bin/plot-llstat &lt;- perl ./usr/bin/llobdstat &lt;- perl </pre> </div></div> <p>This work is looking like it is overlapping with <a href="https://jira.whamcloud.com/browse/LU-3957" title="Create separate server and client binary rpms from one build" class="issue-link" data-issue-key="LU-3957">LU-3957</a></p> <p>The ll_decide_filter_fid and llog_reader tools are also only useful for the server. I think routerstat is useful in clients also, but not sure. </p> <p>As for llstat and llobdstat, I <em>think</em> that those used to be written in bash a long time ago, but it would need some investigation in old versions of Lustre. </p> <p>John Hammond just submitted a patch; <a href="http://review.whamcloud.com/#/c/10510" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/10510</a>; under <a href="https://jira.whamcloud.com/browse/LU-2675" title="clang: code cleanups for sparse static analyzer" class="issue-link" data-issue-key="LU-2675"><del>LU-2675</del></a> that removes a bunch of unused scripts. Looks like this also over laps this work.</p> <p>Thanks. Yes, it overlaps.</p> <p>There could certainly be some overlap, but they are mostly distinct issues. This ticket deals with moving binaries mostly among the existing sub-packages. <a href="https://jira.whamcloud.com/browse/LU-3947" title="Unable to start OSTs when specifying multiple networks in NIDs leads." class="issue-link" data-issue-key="LU-3947"><del>LU-3947</del></a> would create a new sub-package altogether. To my knowledge, no one is currently working on <a href="https://jira.whamcloud.com/browse/LU-3947" title="Unable to start OSTs when specifying multiple networks in NIDs leads." class="issue-link" data-issue-key="LU-3947"><del>LU-3947</del></a>, so the risk of conflicts is low.</p> <blockquote><p>Great. Thanks for comments. Is it Ok to have the following list of files in client package?</p></blockquote> <p>Many of those commands should be installed on both client and servers nodes, e.g. lfs, lctl. At the very lest, these should only be installed as part of a server specific package:</p> <ul> <li>/usr/sbin/ll_decode_filter_fid</li> <li>/usr/sbin/lshowmount (although this is likely to be removed altogether under <a href="http://review.whamcloud.com/10510/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10510/</a>)</li> <li>/etc/ldev.conf</li> <li>/etc/udev/rules.d/99-lustre.rules</li> <li>/usr/sbin/ldev</li> </ul> <p>Patch landed to master.</p> <p>Gerrit change #10654 for this ticket breaks <a href="https://jira.whamcloud.com/browse/LU-1032" title="Add dkms support for kernel modules" class="issue-link" data-issue-key="LU-1032"><del>LU-1032</del></a>/<a href="https://jira.whamcloud.com/browse/LU-5465" title="Enhancements to Lustre DKMS RPM" class="issue-link" data-issue-key="LU-5465"><del>LU-5465</del></a> creation of Lustre Server DKMS RPMs, because now mount_osd_<span class="error">&#91;zfs,ldiskfs&#93;</span>.so lib is part of lustre-osd-<span class="error">&#91;zfs,ldiskfs&#93;</span> binary RPM but is not part/built of/by DKMS RPM.<br/> We need to find a way to fix this, or this will lead to Lustre DKMS Server modules RPM to be unusable, at least for IEEL deployments ...</p> <p>I wrongly suspected this ticket to cause some incmpatibilities, resolving again, sorry.</p> Related LU-3957 Development Rank 1|hzwn3j: Rank (Obsolete) 14084