TruncProof: A Guardrail for LLM-based JSON Generation under Token-Length Constraints

TL;DR

TruncProof is a new method that ensures large language models generate valid JSON outputs within strict token limits, preventing truncation and system errors.

Contribution

It introduces a grammar-constrained decoding approach using LL(1) parsers to enforce token limits while maintaining output validity.

Findings

Successfully generates syntactically correct JSON under strict token constraints.

Combines with advanced decoding to improve semantic accuracy.

Prevents infinite or truncated outputs in LLM-based JSON generation.

Abstract

The LLM-based generation of machine-readable outputs such as JSON has attracted significant attention for integration with external systems. However, existing approaches cannot strictly enforce the maximum number of tokens to be generated, leading to infinite generation or truncated outputs that cause a system malfunction. To address this limitation, we propose TruncProof, a novel grammar-constrained generation method that enables LLMs to produce grammatically valid JSONs while adhering to a predefined token limit. By leveraging the properties of LL(1) parsers, TruncProof efficiently approximates the minimum number of tokens required to complete a grammatically valid output at each decoding step. Experiments on the Text-to-JSON instruction tasks demonstrate that TruncProof successfully generates syntactically correct outputs even under strict token constraints. Furthermore, we show that TruncProof can be effectively combined with advanced decoding strategies, resulting in outputs that are not only grammatically valid but also semantically accurate.