[LU-6246] Add a JSON decoder

• Type: New Feature
• Resolution: Won't Do
• Priority: Minor
• Affects Version/s: Lustre 2.8.0
• Assignee: WC Triage
• Reporter: Frank Zago (Inactive)

Description

For the HSM migration enhancement (LU-6081), it would be nice to be able to transfer data (stripe informaton and mdt_index) between lfs and the copytool in an exchange format, namely JSON.

The current liblustreapi_json.c can only encode JSON. Use the CCAN (Comprehensive C Archive Network) JSON which it relatively small, can encode/decode, and has a decent testsuite.

Replace liblustreapi_json.c with the new implementation.

Issue Links

• is blocking: LU-6081 hsm: add file migrate support
• is related to: LU-9897 Tracking of lustre library improvements.

Comments

[James A Simmons]

Will use YAML instead.

[Nikitas Angelinas]

Hi James. Yes, I think so as well, unfortunately. This would probably have been useful to land, but as you had suggested, since libyaml is now a dependency of Lustre it makes sense to use that instead.

[James A Simmons]

I see the latest LU-6081 work has moved from using JSON to YAML. Since this is the case should we close this ticket and abandon the patches related to this work?

[James A Simmons]

Once LU-9324 lands liblustreapi.so will be supporting YAML. Since this is the case I don't think we need to look at adding a JSON decoder. Instead we need to look at using libYAML to handle JSON. Should we close this ticket or look to changing the direction?

[Frank Zago (Inactive)]

Hi James,
Was there any discussion/decision on that subject at LUG?

[Frank Zago (Inactive)]

I did a quick test to compare the CCAN JSON and the JSON-C (0.11-3 as provided by Ubuntu 12.04).

The test decodes a 1290 bytes strings 1000000 times in a loop.

CCAN JSON does it in 8.58s and JSON-C does it in 20.70s, so it's about 2.4 times slower.

On the other hand, that's still fast enough for the usage being done here, and JSON-C is present in CentOS 6 & 7 and Debian/Ubuntu. The API looks also similar.

So I wouldn't mind dropping these patches if JSON-C has better chance of being used. If JSON-C turns out to be a bottleneck, we can replace it.

[James A Simmons]

Yes.

[Frank Zago (Inactive)]

That one: https://github.com/json-c/json-c ?

[James A Simmons]

Hi Frank. With LUG coming up I like to discuss on developer's day about using JSON in replacement of YAML. Especially since json-c seems to be the standard now.

[Frank Zago (Inactive)]

The licenses used by CCAN JSON are:

• Test anything protocol - tap.c: FreeBSD license (aka 2 clauses BSD)
• Test anything protocol - compiler.h dependency: CC0 (Creative Commons 0)
• JSON: MIT

All 3 licenses are GPLv2 and v3 compatible.