https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9897 Lustre <p>Currently lustre has a large collection of user space libraries: </p> <p>libcfs.a<br/> libcfsutil.a<br/> libptlctl.a<br/> liblnetconfig (optional currently)<br/> liblustreapi.so</p> <p>We really only need two libraries. One being liblnetconfg and the second liblustreapi.so. libptlctl.a should be replaced by liblnetconfig at this point and if we link liblustreapi.so into liblnetconfig.so then we don't need to embedded libcfs*.a into both libraries anymore.</p> <p>Along side this issue is the libyaml dependency. One project was going to introduce JSON support for HSM and then discussion happened at one LUG developers day if we should also move liblnetconfig to libjson as well. So the question becomes do we support both JSON and YAML or drop one of them. YAML/JSON also looks to becoming a hard requirement with some purposed work for using it for setting striping in liblustreapi. </p> <p>Lastly is tracking of the work to change the license to LGPL.</p> Lustre utilities LU-9897

Tracking of lustre library improvements.

Task Minor Open Unresolved James A Simmons James A Simmons always_except Mon, 21 Aug 2017 13:39:03 +0000 Wed, 8 Dec 2021 18:50:44 +0000 Lustre 2.11.0 0 6 <p>Currently all of Lustre uses YAML. We shouldn't be moving only the lnetconfig stuff to JSON. </p> <p>Also what are the justification of moving away from YAML? It's very simple in the way it's being used in both LNet config and Lustre. So my question is it really worth moving all the existing tested code to JSON? or should HSM simply use YAML?</p> <p>Not to mention the backwards compatibility issues when you remove YAML support (YAML is being used in LNet config from 2.7 and earlier for luster). Are we then going to keep both? My vote is to get HSM to use YAML. Are there technical reasons it can't?</p> <p>Another issue that might not be visible externally yet, is that there are 100s of tests that test DLC, mulit-rial, dynamic discovery, which I'm currently working on integrating into the lustre/tests. They all use YAML. We're talking about a significant effort there to move all those tests to JSON.</p> <p>Please read the thread in <a href="https://jira.whamcloud.com/browse/LU-6081" title="hsm: add file migrate support" class="issue-link" data-issue-key="LU-6081">LU-6081</a>. Lets see if libjyaml can now address those issues to consider moving HSM to YAML. I just like to see what the path forward is. One issue which bites people is that the build process for lnet will complete when libyaml-devel is missing. People will build lustre and scratch their head wondering why lnetctl doesn't build. At this point libyaml MUST become a hard requirement.</p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/28752" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28752</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: simplify lustre's libraries<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8e1aa82c2ef9634cf42416bb4a09bb3b0b28406b</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/28752/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28752/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: remove libcfsutils.a and libptlctl.a<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: cc95b58e835a7c30bebef2cf930913ac8152997e</p> <p>Still more work to be done so please keep this open.</p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/30204" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30204</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: make liblnetconfig a hard requirment<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: f899fbd31c7adbc58552d2b076563784213f050e</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/30204/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30204/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: make liblnetconfig a hard requirment<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: ccafcc7116411c7d276c5b8523b0effb4c4dd796</p> <p>Hi James,</p> <p>just wanted to compile Lustre client with dlc support and saw the option is removed and by default activated when yaml-devel is found. Shouldn't <em>--disable/enable-dlc</em> be also removed here:</p> <p> </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">&gt;grep -r <span class="code-quote">"disable-dlc"</span> . ./lustre-dkms_pre-build.sh: --disable-manpages --disable-dlc ./config/lustre-build.m4: --enable-dlc | --disable-dlc ) ;; </pre> </div></div> <p> </p> <p>Cheers Thomas</p> <p> </p> <p>Ugh, the disable-dlc was missed. I will submit a patch which will fix this. </p> <p>The disable-dlc issues have been fixed with the patch landed for <a href="https://jira.whamcloud.com/browse/LU-5541" title="improve liblustreapi build" class="issue-link" data-issue-key="LU-5541"><del>LU-5541</del></a>. Also we now have two librariess that applications can link to, liblnetconfig and  liblustreapi.so. No more needing to link to libcfs libraries. Much more sane now. Still some more work to be done.</p> <p>James Simmons (uja.ornl@yahoo.com) uploaded a new patch: <a href="https://review.whamcloud.com/35422" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35422</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> zfs: migrate autoconf detection to pkgconfig<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 66328e1aa2e5c1f257d38625be071a0014a6d588</p> <p>James Simmons (jsimmons@infradead.org) uploaded a new patch: <a href="https://review.whamcloud.com/35531" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35531</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> build: use pkgconf for detecting libblkid<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 51730b2573cf294f776176accda74b48d9f1e64f</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/35531/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35531/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> build: use pkgconf for detecting libblkid<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 6bc3237477069c8ddc70deee6e0d7cbbbdf92a08</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36010" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36010</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: remove dependency on libblkid<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8804d466e8e460c0e23a8e1a7bada1a391a0b3ed</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36016" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36016</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: remove llverfs dependency on libext2fs<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: e999d93deef0f61e7d9973d2689508ea09223488</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36010/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36010/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: remove dependency on libblkid<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: fe0e096567ba148d5690316d2bad2226c0cf70bf</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36016/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36016/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: remove llverfs dependency on libext2fs<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 6703ea0bf5bd5584998472cadb6de83211802861</p> <p>James Simmons (jsimmons@infradead.org) uploaded a new patch: <a href="https://review.whamcloud.com/38825" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38825</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> build: add binaries to .gitignore<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: d67b69dfaff005fec7e6a8854a0c3ebc5c67fda0</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38825/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38825/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> build: add binaries to .gitignore<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 60df7a8845859639d16f1c2163a65a74a6a51f78</p> <p>James Simmons (jsimmons@infradead.org) uploaded a new patch: <a href="https://review.whamcloud.com/38921" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38921</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: have lfs.c use lstddef.h<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 1ea538fb5e0b657dd7ec8ee962d527f54f0f5e03</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38921/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38921/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: have lfs.c use lstddef.h<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b12c5467366f1ce66bf8fe9973803414c947c322</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/39347" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39347</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: allow setting llverfs subdir count<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 494a64527faf34660fc8de824579f6958243bb34</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/39347/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/39347/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> utils: allow setting llverfs subdir count<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: c0086134f7ac8659422176a3db790a079632a8ea</p> <p>"James Simmons &lt;jsimmons@infradead.org&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/44778" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44778</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> tests: add generated files to .gitignore<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: c4db303f1422879ac06e3c09d134ed3452af3b3e</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/44778/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44778/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> tests: add generated files to .gitignore<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3ad89522912d74ec1c854e449957c45d1285cb25</p> <p>"James Simmons &lt;jsimmons@infradead.org&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/45793" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/45793</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9897" title="Tracking of lustre library improvements." class="issue-link" data-issue-key="LU-9897">LU-9897</a> build: create kmod-lnet-devel<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: b92e28bd904d6d790e6abd6093651eec3418cff3</p> Related LU-11742 LU-8854 LU-10717 LU-6246 LU-9324 LU-5969 LU-6081 LU-5541 LU-13315 LU-10556 LU-12214 LU-12720 LU-9215 LU-9898 Development Rank 1|hzziq7: Rank (Obsolete) 9223372036854775807