Generative Distribution Prediction: A Unified Approach to Multimodal Learning

TL;DR

The paper introduces Generative Distribution Prediction (GDP), a versatile framework that uses multimodal synthetic data generation to improve predictive accuracy across various data types and applications.

Contribution

GDP is a novel, model-agnostic framework that leverages generative models like diffusion models for multimodal prediction with theoretical guarantees.

Findings

GDP improves predictive accuracy across diverse tasks.

It is compatible with any high-fidelity generative model.

Empirical results show versatility in multiple domains.

Abstract

Accurate prediction with multimodal data-encompassing tabular, textual, and visual inputs or outputs-is fundamental to advancing analytics in diverse application domains. Traditional approaches often struggle to integrate heterogeneous data types while maintaining high predictive accuracy. We introduce Generative Distribution Prediction (GDP), a novel framework that leverages multimodal synthetic data generation-such as conditional diffusion models-to enhance predictive performance across structured and unstructured modalities. GDP is model-agnostic, compatible with any high-fidelity generative model, and supports transfer learning for domain adaptation. We establish a rigorous theoretical foundation for GDP, providing statistical guarantees on its predictive accuracy when using diffusion models as the generative backbone. By estimating the data-generating distribution and adapting to various loss functions for risk minimization, GDP enables accurate point predictions across multimodal settings. We empirically validate GDP on four supervised learning tasks-tabular data prediction, question answering, image captioning, and adaptive quantile regression-demonstrating its versatility and effectiveness across diverse domains.