https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5541 Lustre <p>Currently liblustreapi export all its symbols to the application, some of which are undesirable, such as do_ioctl, dump, ... Use the linker to clean that up. </p> <p>Use libtool to build the shared library.</p> LU-5541

improve liblustreapi build

Improvement Minor Resolved Fixed Nathaniel Clark Frank Zago patch Mon, 25 Aug 2014 15:35:37 +0000 Mon, 31 Jan 2022 05:02:33 +0000 Thu, 25 Jan 2018 05:12:30 +0000 Lustre 2.11.0 0 8 <p>(abandonned) <a href="http://review.whamcloud.com/11581" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11581</a><br/> (abandonned) <a href="http://review.whamcloud.com/11582" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11582</a><br/> (abandonned) <a href="http://review.whamcloud.com/11583" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11583</a><br/> (abandonned) <a href="http://review.whamcloud.com/11584" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11584</a></p> <p>Somewhat related to these patches, there are several archives that are installed by the rpms that look unnecessary:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>/usr/lib64/libcfsutil.a /usr/lib64/libiam.a /usr/lib64/liblustreapi.a /usr/lib64/libptlctl.a </pre> </div></div> <p>If used, these should probably be replaced by a shared library.</p> <p><a href="http://review.whamcloud.com/11625" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11625</a><br/> <a href="http://review.whamcloud.com/11643" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11643</a></p> <p>With the base patch worked out we should discuss what functions to export. I'm in agreement with Henri that all functions to exports should began with llapi_* for liblustreapi. Anyone disagree?</p> <p>There is some functions that start with llapi_* that shouldn't be exported (like llapi_printf, llapi_chomp_string, all the json functions, ...)</p> <p>Frank do you have a list of llapi_* functions not to export.</p> <p>No, but you can modify the linker script to not export anything, then add the missing functions as the linker fails to link the various programs.</p> <p>You can then compare that list with the list of all llapi functions obtained previously with</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>nm liblustreapi.so | grep " T llapi_" </pre> </div></div> <p>However I believe that the function names should be whitelisted instead of blacklisting some names.</p> <p>I'm seeing this when building on Ubuntu and only on Ubuntu.</p> <p>gcc -DHAVE_CONFIG_H -I. -I../.. -include /tmp/jsimmons-ppc/rpmbuild/usr/src/lustre-2.7.65/debian/tmp/modules-deb/usr<br/> _src/modules/lustre/undef.h -include /tmp/jsimmons-ppc/rpmbuild/usr/src/lustre-2.7.65/debian/tmp/modules-deb/usr_src/m<br/> odules/lustre/config.h -I/tmp/jsimmons-ppc/rpmbuild/usr/src/lustre-2.7.65/debian/tmp/modules-deb/usr_src/modules/lustr<br/> e/libcfs/include -I/tmp/jsimmons-ppc/rpmbuild/usr/src/lustre-2.7.65/debian/tmp/modules-deb/usr_src/modules/lustre/lnet/include -I/tmp/jsimmons-ppc/rpmbuild/usr/src/lustre-2.7.65/debian/tmp/modules-deb/usr_src/modules/lustre/lustre/include -fPIC -D_GNU_SOURCE -D_LARGEFILE64_SOURCE=1 -D_FILE_OFFSET_BITS=64 -DLUSTRE_UTILS=1 -I/opt/lam/include -g -O2 -c -o multiop.o multiop.c<br/> make<span class="error">&#91;7&#93;</span>: *** No rule to make target `../../lustre/utils/liblustreapi.la', needed by `multiop'. Stop.<br/> make<span class="error">&#91;7&#93;</span>: *** Waiting for unfinished jobs....</p> <p>It only happens there. Frank have any ideas why?</p> <p>I guess it's because it's a more recent version of autoconf than elsewhere.</p> <p>James, I have built lustre on Ubuntu 15.10 with the 2 patches above plus <a href="http://review.whamcloud.com/#/c/17581" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/17581</a>, with no problem.</p> <p>Did you do a make deb as well? Doing make does work but it fails for make debs. I do suspect it is a setup issue.</p> <p>No, I didn't do that. It's possible that some files are missing in automake. For instance, if you do a "make dist", can you build from the generated tar file?</p> <p>I have attempted to build on my Ubuntu 15 home box but now it will no longer build due to missing keys. As for our Ubuntu14 platform on Power8 I just think it is totally messed up with the dpkg system. It can't find the liblustreapi.la so it failed to build. This might be specific to our Power8 platform but in any case our admins are working hard to get as far away from Ubuntu as possible to using RHEL7. So by the time this patch lands it will be a non-issue. With all the problems we should declare Ubuntu support dead.</p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/28701" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28701</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> test: add proper libtool setup for test application<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5ec4a652ea139d2c4960588e4282a6c655e777ef</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/11625/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/11625/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> build: build static and dynamic liblustreapi<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: ab1df50e73ff838053fff62302c3b884e4e19552</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/11643/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/11643/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> lustreapi: only export the API symbols<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b36c377ff25c20417c481eab3798e67d042ec3a3</p> <p>with this patch landed I can't run from the build tree (as host and VM has different paths):</p> <ol> <li>/work/head2/lustre/utils/lctl<br/> /work/head2/lustre/utils/lctl: line 202: cd: /home/alexey/head2/lustre/utils: No such file or directory<br/> gcc: lustre_lfsck.o: No such file or directory<br/> gcc: obd.o: No such file or directory<br/> gcc: lustre_cfg.o: No such file or directory<br/> gcc: lctl.o: No such file or directory<br/> gcc: lsnapshot.o: No such file or directory<br/> gcc: ./.libs/liblustreapi.so: No such file or directory</li> </ol> <p>You need to run autogen.sh; ./configure .... Oleg works like this as well and I made sure he could continue his work flow.</p> <p>hmm, that didn't work for me. just tried.</p> <p>Does it work with a fresh checkout? What is your environment you are working in?</p> <p>got the same with a fresh checkout.</p> <p>--disable-shared worked for me, thanks.</p> <p>AH so you are doing the same environment as Oleg. As a note to people spawning VMs that are read only like Oleg does. You will need to build lustre with ./configure --disable-shared. Since it looks like it is working for Alex we can close this ticket. Future work will be done under <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a></p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/28782" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28782</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> lustreapi: only export the API symbols"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: cb8908f7bfa6a4ed662646111553769a2aeff764</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/28783" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28783</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> build: build static and dynamic liblustreapi"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 422457d0bdc180709ecf734de85fd36adc229c77</p> <p>So does anyone understand how the debian/ubuntu version of libtool works. Its totally different than other systems.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/28782/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28782/</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> lustreapi: only export the API symbols"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: fe4b7e75c0b65d6e1099e6043c7d21637f87403e</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/28783/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28783/</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> build: build static and dynamic liblustreapi"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: e1679d0348856bc68589493e7ea86f1aaf21e1af</p> <p>BTW just to let people know building Ubuntu 16 is very broken in general for lustre <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/sad.png" height="16" width="16" align="absmiddle" alt="" border="0"/> </p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/30562" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30562</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> build: move libcfs and liblustreapi over to libtool<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: c672448588f55f6dbce5351c2f3033ecf57835e2</p> <p>Okay I managed to get everything to build on Ubuntu17 but now it fails on Ubuntu16. I'm going to need help from debian people here to get that working.</p> <p>Thanks to Thomas Stibor the Ubuntu/Debian build issues have been resolved.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/30562/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30562/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> build: move libcfs and liblustreapi over to libtool<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 7b659a1e68174212c443761be1880774b440a901</p> <p>So is that patch just landed to master the last part of this work? Can the ticket be marked as resolved?</p> <p>One patch left</p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/30865" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30865</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> lustreapi: only export the API symbols<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 22ea21137a936e906171bbc073dfee9767fd24d2</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/30865/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30865/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a> lustreapi: only export the API symbols<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: e954d93f264d8ae559cbd41ed6d7b977674dd42b</p> <p>The end of an era <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Yes. Libraries are sane now!!!</p> Related LU-10514 LU-6243 LU-3074 LU-7356 LU-9897 Development Rank 1|hzwufr: Rank (Obsolete) 15425