Learning Stochastic Parametric Differentiable Predictive Control Policies

TL;DR

This paper introduces SP-DPC, a scalable neural control policy learning method for stochastic linear systems with chance constraints, using automatic differentiation and theoretical guarantees for stability and constraint satisfaction.

Contribution

The paper presents SP-DPC, a novel deterministic approximation approach enabling scalable, model-based neural policy learning for stochastic systems with probabilistic constraints.

Findings

SP-DPC efficiently learns control policies for high-dimensional stochastic systems.

The method provides probabilistic guarantees on stability and constraint satisfaction.

Numerical examples demonstrate scalability and effectiveness of SP-DPC.

Abstract

The problem of synthesizing stochastic explicit model predictive control policies is known to be quickly intractable even for systems of modest complexity when using classical control-theoretic methods. To address this challenge, we present a scalable alternative called stochastic parametric differentiable predictive control (SP-DPC) for unsupervised learning of neural control policies governing stochastic linear systems subject to nonlinear chance constraints. SP-DPC is formulated as a deterministic approximation to the stochastic parametric constrained optimal control problem. This formulation allows us to directly compute the policy gradients via automatic differentiation of the problem's value function, evaluated over sampled parameters and uncertainties. In particular, the computed expectation of the SP-DPC problem's value function is backpropagated through the closed-loop system rollouts parametrized by a known nominal system dynamics model and neural control policy which allows for direct model-based policy optimization. We provide theoretical probabilistic guarantees for policies learned via the SP-DPC method on closed-loop stability and chance constraints satisfaction. Furthermore, we demonstrate the computational efficiency and scalability of the proposed policy optimization algorithm in three numerical examples, including systems with a large number of states or subject to nonlinear constraints.