The Behavioral Diversity of Java JSON Libraries

TL;DR

This study systematically analyzes 20 Java JSON libraries, revealing significant behavioral diversity especially with ill-formed files, and suggests a multi-version approach for robust processing.

Contribution

First comprehensive analysis of input/output behavior of Java JSON libraries, highlighting diversity and implications for software engineering.

Findings

High behavioral diversity with ill-formed files

Differences in data structure choices and number encoding

Potential for multi-version architectures for robustness

Abstract

JSON is an essential file and data format in do-mains that span scientific computing, web APIs or configuration management. Its popularity has motivated significant software development effort to build multiple libraries to process JSON data. Previous studies focus on performance comparison among these libraries and lack a software engineering perspective.We present the first systematic analysis and comparison of the input / output behavior of 20 JSON libraries, in a single software ecosystem: Java/Maven. We assess behavior diversity by running each library against a curated set of 473 JSON files, including both well-formed and ill-formed files. The main design differences, which influence the behavior of the libraries, relate to the choice of data structure to represent JSON objects and to the encoding of numbers. We observe a remarkable behavioral diversity with ill-formed files, or corner cases such as large numbers or duplicate data. Our unique behavioral assessment of JSON libraries paves the way for a robust processing of ill-formed files, through a multi-version architecture.