Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Data

TL;DR

This paper introduces PATH, a novel framework using privately fine-tuned large language models to generate synthetic longitudinal tabular data, effectively preserving temporal dynamics and dependencies that traditional methods fail to maintain.

Contribution

The paper presents PATH, a new generative approach that models entire user histories as sequences, capturing temporal dependencies in private synthetic data generation.

Findings

PATH reduces distributional distance to real data by over 60%

PATH decreases state transition errors by nearly 50%

PATH achieves similar marginal fidelity to existing methods

Abstract

Research on differentially private synthetic tabular data has largely focused on independent and identically distributed rows where each record corresponds to a unique individual. This perspective neglects the temporal complexity in longitudinal datasets, such as electronic health records, where a user contributes an entire (sub) table of sequential events. While practitioners might attempt to model such data by flattening user histories into high-dimensional vectors for use with standard marginal-based mechanisms, we demonstrate that this strategy is insufficient. Flattening fails to preserve temporal coherence even when it maintains valid marginal distributions. We introduce PATH, a novel generative framework that treats the full table as the unit of synthesis and leverages the autoregressive capabilities of privately fine-tuned large language models. Extensive evaluations show that PATH effectively captures long-range dependencies that traditional methods miss. Empirically, our method reduces the distributional distance to real trajectories by over 60% and reduces state transition errors by nearly 50% compared to leading marginal mechanisms while achieving similar marginal fidelity.