Towards Verifiable Text Generation with Symbolic References

TL;DR

This paper introduces SymGen, a method that interleaves symbolic references within LLM outputs to improve verifiability and reduce manual verification effort, maintaining fluency and factuality.

Contribution

SymGen is a novel approach that enables LLMs to include explicit symbolic references in their outputs for easier validation.

Findings

LLMs can generate text with accurate symbolic references.

Symbolic references aid manual verification of generated text.

The approach maintains fluency and factuality of outputs.

Abstract

LLMs are vulnerable to hallucinations, and thus their outputs generally require laborious human verification for high-stakes applications. To this end, we propose symbolically grounded generation (SymGen) as a simple approach for enabling easier manual validation of an LLM's output. SymGen prompts an LLM to interleave its regular output text with explicit symbolic references to fields present in some conditioning data (e.g., a table in JSON format). The references can be used to display the provenance of different spans of text in the generation, reducing the effort required for manual verification. Across a range of data-to-text and question-answering experiments, we find that LLMs are able to directly output text that makes use of accurate symbolic references while maintaining fluency and factuality. In a human study we further find that such annotations can streamline human verification of machine-generated text. Our code will be available at http://symgen.github.io.