Generative Models in Decision Making: A Survey

TL;DR

This survey reviews how generative models have transformed decision-making by enabling high-fidelity trajectory generation and discusses a unified probabilistic framework for their roles, applications, and challenges in high-stakes domains.

Contribution

It introduces a function-centric taxonomy of generative decision models within Control as Inference, unifying diverse approaches and analyzing systemic risks and future challenges.

Findings

Four functional roles: Controllers, Modelers, Optimizers, Evaluators.

Critical analysis of generative families across dimensions.

Discussion of systemic risks like dynamics hallucination.

Abstract

Generative models have fundamentally reshaped the landscape of decision-making, reframing the problem from pure scalar reward maximization to high-fidelity trajectory generation and distribution matching. This paradigm shift addresses intrinsic limitations in classical Reinforcement Learning (RL), particularly the limited expressivity of standard unimodal policy distributions in capturing complex, multi-modal behaviors embedded in diverse datasets. However, current literature often treats these models as isolated algorithmic improvements, rarely synthesizing them into a single comprehensive framework. This survey proposes a principled taxonomy grounding generative decision-making within the probabilistic framework of Control as Inference. By performing a variational factorization of the trajectory posterior, we conceptualize four distinct functional roles: Controllers for amortized policy inference, Modelers for dynamics priors, Optimizers for iterative trajectory refinement, and Evaluators for trajectory guidance and value assessment. Unlike existing architecture-centric reviews, this function-centric framework allows us to critically analyze representative generative families across distinct dimensions. Furthermore, we examine deployment in high-stakes domains, specifically Embodied AI, Autonomous Driving, and AI for Science, highlighting systemic risks such as dynamics hallucination in world models and proxy exploitation. Finally, we chart the path toward Generalist Physical Intelligence, identifying pivotal challenges in inference efficiency, trustworthiness, and the emergence of Physical Foundation Models.