https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-6246 Lustre <p>For the HSM migration enhancement (<a href="https://jira.whamcloud.com/browse/LU-6081" title="hsm: add file migrate support" class="issue-link" data-issue-key="LU-6081">LU-6081</a>), it would be nice to be able to transfer data (stripe informaton and mdt_index) between lfs and the copytool in an exchange format, namely JSON.</p> <p>The current liblustreapi_json.c can only encode JSON. Use the CCAN (Comprehensive C Archive Network) JSON which it relatively small, can encode/decode, and has a decent testsuite.</p> <p>Replace liblustreapi_json.c with the new implementation.</p> LU-6246

Add a JSON decoder

New Feature Minor Resolved Won't Do WC Triage Frank Zago Fri, 13 Feb 2015 23:20:11 +0000 Fri, 2 Nov 2018 00:27:22 +0000 Fri, 2 Nov 2018 00:27:22 +0000 Lustre 2.8.0 0 8 <p>frank zago (fzago@cray.com) uploaded a new patch: <a href="http://review.whamcloud.com/13770" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13770</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-6246" title="Add a JSON decoder" class="issue-link" data-issue-key="LU-6246"><del>LU-6246</del></a> utils: add CCAN JSON parser / emitter from<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: fb7008342fa71cc919a1c743f304fa053fdaac65</p> <p>frank zago (fzago@cray.com) uploaded a new patch: <a href="http://review.whamcloud.com/13771" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13771</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-6246" title="Add a JSON decoder" class="issue-link" data-issue-key="LU-6246"><del>LU-6246</del></a> tests: add CCAN Test Framework from CCAN<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 0db83eecf2101326b1e7c7ea23a13f8e3f94646f</p> <p>frank zago (fzago@cray.com) uploaded a new patch: <a href="http://review.whamcloud.com/13772" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13772</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-6246" title="Add a JSON decoder" class="issue-link" data-issue-key="LU-6246"><del>LU-6246</del></a> tests: integrate CCAN JSON<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a46eae5c5712bfd6be1071f999a42a1464a59638</p> <p>frank zago (fzago@cray.com) uploaded a new patch: <a href="http://review.whamcloud.com/13773" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13773</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-6246" title="Add a JSON decoder" class="issue-link" data-issue-key="LU-6246"><del>LU-6246</del></a> tests: split JSON_NUMBER into int and float<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 13a952f777981a83de7b0dc1068a9c48d813004b</p> <p>frank zago (fzago@cray.com) uploaded a new patch: <a href="http://review.whamcloud.com/13774" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13774</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-6246" title="Add a JSON decoder" class="issue-link" data-issue-key="LU-6246"><del>LU-6246</del></a> user: replace liblustreapi JSON<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 910e31fe91844a4308e839736c07798c3ff8f15d</p> <p>The licenses used by CCAN JSON are:</p> <ul class="alternate" type="square"> <li>Test anything protocol - tap.c: FreeBSD license (aka 2 clauses BSD)</li> <li>Test anything protocol - compiler.h dependency: CC0 (Creative Commons 0)</li> <li>JSON: MIT</li> </ul> <p>All 3 licenses are GPLv2 and v3 compatible.</p> <p>Hi Frank. With LUG coming up I like to discuss on developer's day about using JSON in replacement of YAML. Especially since json-c seems to be the standard now.</p> <p>That one: <a href="https://github.com/json-c/json-c" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/json-c/json-c</a> ?</p> <p>Yes.</p> <p>I did a quick test to compare the CCAN JSON and the JSON-C (0.11-3 as provided by Ubuntu 12.04).</p> <p>The test decodes a 1290 bytes strings 1000000 times in a loop.</p> <p>CCAN JSON does it in 8.58s and JSON-C does it in 20.70s, so it's about 2.4 times slower.</p> <p>On the other hand, that's still fast enough for the usage being done here, and JSON-C is present in CentOS 6 &amp; 7 and Debian/Ubuntu. The API looks also similar.</p> <p>So I wouldn't mind dropping these patches if JSON-C has better chance of being used. If JSON-C turns out to be a bottleneck, we can replace it.</p> <p>Hi James,<br/> Was there any discussion/decision on that subject at LUG?</p> <p>Once <a href="https://jira.whamcloud.com/browse/LU-9324" title="sanity-pfl test 10 needs to reset the file system default layout" class="issue-link" data-issue-key="LU-9324"><del>LU-9324</del></a> lands liblustreapi.so will be supporting YAML. Since this is the case I don't think we need to look at adding a JSON decoder. Instead we need to look at using libYAML to handle JSON. Should we close this ticket or look to changing the direction?</p> <p>I see the latest <a href="https://jira.whamcloud.com/browse/LU-6081" title="hsm: add file migrate support" class="issue-link" data-issue-key="LU-6081">LU-6081</a> work has moved from using JSON to YAML. Since this is the case should we close this ticket and abandon the patches related to this work?</p> <p>Hi James. Yes, I think so as well, unfortunately. This would probably have been useful to land, but as you had suggested, since libyaml is now a dependency of Lustre it makes sense to use that instead.</p> <p>Will use YAML instead.</p> Blocker LU-6081 Related LU-9897 Development Rank 1|hzx6d3: Rank (Obsolete) 17495