Large Language Models Based JSON Parser Fuzzing for Bug Discovery and Behavioral Analysis

TL;DR

This paper explores using Large Language Models to generate test cases and mutants for JSON parsers, aiming to discover bugs and analyze behavioral differences among open-source implementations.

Contribution

It introduces a novel approach leveraging LLMs for automated JSON parser fuzzing and behavioral analysis, which has not been extensively studied before.

Findings

LLMs can generate effective test cases for JSON parsers

Behavioral diversities among parsers can be identified using LLM-generated mutants

Potential bugs in open-source JSON parsers can be uncovered

Abstract

Fuzzing has been incredibly successful in uncovering bugs and vulnerabilities across diverse software systems. JSON parsers play a vital role in modern software development, and ensuring their reliability is of great importance. This research project focuses on leveraging Large Language Models (LLMs) to enhance JSON parser testing. The primary objectives are to generate test cases and mutants using LLMs for the discovery of potential bugs in open-source JSON parsers and the identification of behavioral diversities among them. We aim to uncover underlying bugs, plus discovering (and overcoming) behavioral diversities.