Foundation Models for Decision Making: Problems, Methods, and Opportunities

TL;DR

This paper reviews how foundation models are evolving to support decision making in complex, real-world applications by integrating interaction, reasoning, and planning capabilities across diverse domains.

Contribution

It provides a comprehensive overview of recent methods and challenges in applying foundation models to decision making tasks across multiple disciplines.

Findings

Foundation models are increasingly used for decision making in real-world environments.

Recent approaches include prompting, generative modeling, planning, and reinforcement learning.

Open problems involve grounding models in practical decision contexts.

Abstract

Foundation models pretrained on diverse data at scale have demonstrated extraordinary capabilities in a wide range of vision and language tasks. When such models are deployed in real world environments, they inevitably interface with other entities and agents. For example, language models are often used to interact with human beings through dialogue, and visual perception models are used to autonomously navigate neighborhood streets. In response to these developments, new paradigms are emerging for training foundation models to interact with other agents and perform long-term reasoning. These paradigms leverage the existence of ever-larger datasets curated for multimodal, multitask, and generalist interaction. Research at the intersection of foundation models and decision making holds tremendous promise for creating powerful new systems that can interact effectively across a diverse range of applications such as dialogue, autonomous driving, healthcare, education, and robotics. In this manuscript, we examine the scope of foundation models for decision making, and provide conceptual tools and technical background for understanding the problem space and exploring new research directions. We review recent approaches that ground foundation models in practical decision making applications through a variety of methods such as prompting, conditional generative modeling, planning, optimal control, and reinforcement learning, and discuss common challenges and open problems in the field.