On Exact Solutions to the Linear Bellman Equation
David Ohlin, Richard Pates, Murat Arcak

TL;DR
This paper derives explicit solutions to the Bellman equation for certain linear and semilinear control systems, enabling distributed computation and extending the class of solvable Markov decision processes.
Contribution
It provides sufficient conditions for explicit Bellman solutions in linear and semilinear systems, reformulates Linearly Solvable MDPs as continuous control problems, and extends previous results to nonlinear input dynamics.
Findings
Explicit solutions for linear control systems.
Extension of solvable MDP class to semilinear dynamics.
Applicability demonstrated in LQR and shortest path scenarios.
Abstract
This paper presents sufficient conditions for optimal control of systems with dynamics given by a linear operator, in order to obtain an explicit solution to the Bellman equation that can be calculated in a distributed fashion. Further, the class of Linearly Solvable MDP is reformulated as a continuous-state optimal control problem. It is shown that this class naturally satisfies the conditions for explicit solution of the Bellman equation, motivating the extension of previous results to semilinear dynamics to account for input nonlinearities. The applicability of the given conditions is illustrated in scenarios with linear and quadratic cost, corresponding to the Stochastic Shortest Path and Linear-Quadratic Regulator problems.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOpinion Dynamics and Social Influence
