Deep Learning for Sequential Decision Making under Uncertainty: Foundations, Frameworks, and Frontiers

TL;DR

This paper discusses how deep learning complements traditional optimization methods in sequential decision-making under uncertainty, emphasizing integration across various AI architectures and operational domains.

Contribution

It provides an OR/MS-centered perspective on combining deep learning with optimization for decision-making under uncertainty, highlighting recent advances and future directions.

Findings

Deep learning enhances scalability and adaptability in decision systems.

Integration of learning and optimization improves decision quality in complex environments.

Emerging applications include supply chains, healthcare, and autonomous systems.

Abstract

Artificial intelligence (AI) is moving increasingly beyond prediction to support decisions in complex, uncertain, and dynamic environments. This shift creates a natural intersection with operations research and management sciences (OR/MS), which have long offered conceptual and methodological foundations for sequential decision-making under uncertainty. At the same time, recent advances in deep learning, including feedforward neural networks, LSTMs, transformers, and deep reinforcement learning, have expanded the scope of data-driven modeling and opened new possibilities for large-scale decision systems. This tutorial presents an OR/MS-centered perspective on deep learning for sequential decision-making under uncertainty. Its central premise is that deep learning is valuable not as a replacement for optimization, but as a complement to it. Deep learning brings adaptability and scalable approximation, whereas OR/MS provides the structural rigor needed to represent constraints, recourse, and uncertainty. The tutorial reviews key decision-making foundations, connects them to the major neural architectures in modern AI, and discusses leading approaches to integrating learning and optimization. It also highlights emerging impact in domains such as supply chains, healthcare and epidemic response, agriculture, energy, and autonomous operations. More broadly, it frames these developments as part of a wider transition from predictive AI toward decision-capable AI and highlights the role of OR/MS in shaping the next generation of integrated learning--optimization systems.