A Unified Framework for Tabular Generative Modeling: Loss Functions, Benchmarks, and Improved Multi-objective Bayesian Optimization Approaches

TL;DR

This paper presents a comprehensive framework for tabular data generative modeling, introducing a new loss function, a multi-objective Bayesian optimization strategy, and extensive benchmarking to improve data fidelity and hyperparameter tuning.

Contribution

It introduces a novel correlation-aware loss function, a multi-objective Bayesian optimization method, and a unified benchmarking framework for tabular generative models.

Findings

The correlation-aware loss improves synthetic data quality and downstream ML performance.

IORBO outperforms standard Bayesian optimization in hyperparameter tuning.

The framework achieves better data fidelity and hyperparameter optimization across 20 datasets.

Abstract

Deep learning (DL) models require extensive data to achieve strong performance and generalization. Deep generative models (DGMs) offer a solution by synthesizing data. Yet current approaches for tabular data often fail to preserve feature correlations and distributions during training, struggle with multi-metric hyperparameter selection, and lack comprehensive evaluation protocols. We address this gap with a unified framework that integrates training, hyperparameter tuning, and evaluation. First, we introduce a novel correlation- and distribution-aware loss function that regularizes DGMs, enhancing their ability to generate synthetic tabular data that faithfully represents the underlying data distributions. Theoretical analysis establishes stability and consistency guarantees. To enable principled hyperparameter search via Bayesian optimization (BO), we also propose a new multi-objective aggregation strategy based on iterative objective refinement Bayesian optimization (IORBO), along with a comprehensive statistical testing framework. We validate the proposed approach using a benchmarking framework with twenty real-world datasets and ten established tabular DGM baselines. The correlation-aware loss function significantly improves synthetic data fidelity and downstream machine learning (ML) performance, while IORBO consistently outperforms standard Bayesian optimization (SBO) in hyperparameter selection. The unified framework advances tabular generative modeling beyond isolated method improvements. Code is available at: https://github.com/vuhoangminh/TabGen-Framework