A Design Space for the Critical Validation of LLM-Generated Tabular Data

TL;DR

This paper introduces a structured design space framework for the critical validation of tabular data generated by large language models, addressing the need for systematic evaluation methods.

Contribution

It defines a two-dimensional design space for validation, maps existing approaches, and analyzes two methods in detail to demonstrate their descriptive capabilities.

Findings

Mapped 19 validation approaches within the design space

Identified key analysis tasks for different validation scenarios

Demonstrated the descriptive power of two selected approaches

Abstract

LLM-generated tabular data is creating new opportunities for data-driven applications in academia, business, and society. To leverage benefits like missing value imputation, labeling, and enrichment with context-aware attributes, LLM-generated data needs a critical validation process. The number of pioneering approaches is increasing fast, opening a promising validation space that, so far, remains unstructured. We present a design space for the critical validation of LLM-generated tabular data with two dimensions: First, the Analysis Granularity dimension: from within-attribute (single-item and multi-item) to across-attribute perspectives (1 x 1, 1 x m, and n x n). Second, the Data Source dimension: differentiating between LLM-generated values, ground truth values, explanations, and their combinations. We discuss analysis tasks for each dimension cross-cut, map 19 existing validation approaches, and discuss the characteristics of two approaches in detail, demonstrating descriptive power.