The maximum principle for discrete-time control systems and applications to dynamic games

TL;DR

This paper establishes a discrete-time maximum principle for optimal control problems, providing necessary and sufficient conditions for optimality, and applies it to derive equilibrium conditions in dynamic games.

Contribution

It introduces a discrete-time maximum principle using Gateaux differentials, extending continuous-time optimal control theory to discrete systems and dynamic games.

Findings

Maximum principle proven for discrete-time systems

Necessary and sufficient conditions for optimality derived

Application to Nash equilibria in dynamic games

Abstract

We study deterministic nonstationary discrete-time optimal control problems in both finite and infinite horizon. With the aid of Gateaux differentials, we prove a discrete-time maximum principle in analogy with the well-known continuous-time maximum principle. We show that this maximum principle, together with a transversality condition, is a necessary condition for optimality; we also show that it is sufficient under additional hypotheses. We use Gateaux differentials as a natural setting to derive first-order conditions. Additionally, we use the discrete-time maximum principle to derive the discrete-time Euler equation and to characterize Nash equilibria for discrete-time dynamic games.