SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generation

TL;DR

SAGE introduces a novel LLM-based framework for generating high-fidelity synthetic tabular data by enforcing sparse, dynamic feature dependencies, improving data utility and reducing policy violations.

Contribution

It proposes a new method that models feature dependencies sparsely and adaptively, addressing limitations of previous dense, static dependency approaches.

Findings

Boosts F1 scores by 10% over previous methods

Reduces policy violations by one point

Improves data fidelity and downstream utility

Abstract

Generating high-fidelity synthetic tabular data remains a critical challenge for enhancing data availability in privacy-sensitive and low-resource domains. Recent approaches leverage LLMs by representing table rows as sequences, yet suffer from two fundamental limitations: (1) they model feature dependencies densely, introducing spurious correlations; and (2) they assume static relationships between features, ignoring how these dependencies vary with feature values. To overcome these limitations, we introduce SAGE (Sparse Adaptive Guidance), a novel LLM-based generation framework that enforces sparse and dynamic dependency guidance. SAGE discretizes features into value-aware pseudo-features and constructs a mutual information-based sparse dependency graph. This graph adaptively guides generation through explicit context selection or implicit logit correction, enabling LLMs to focus on truly relevant information during synthesis. Our extensive experiments across six datasets and multiple tasks reveal that SAGE not only improves data fidelity and downstream utility, boosting F1 scores by 10% compared to previous LLM-based methods, but also reduces policy violations by one point. These results highlight the importance of adaptive structure in tabular data generation and provide new insights into context-sensitive control of LLMs.