Policy Optimization over Submanifolds for Linearly Constrained Feedback Synthesis

TL;DR

This paper introduces a Riemannian manifold-based approach for linearly constrained policy optimization in control systems, providing a Newton-type algorithm with convergence guarantees and demonstrating its effectiveness through numerical examples.

Contribution

It develops a novel geometric framework for constrained policy optimization on the manifold of Schur stabilizing controllers, including a new Newton-type algorithm with convergence guarantees.

Findings

The proposed algorithm converges locally without exponential mapping.

Numerical examples demonstrate improved performance over existing methods.

The framework unifies various constrained control problems under a geometric perspective.

Abstract

In this paper, we study linearly constrained policy optimization over the manifold of Schur stabilizing controllers, equipped with a Riemannian metric that emerges naturally in the context of optimal control problems. We provide extrinsic analysis of a generic constrained smooth cost function, that subsequently facilitates subsuming any such constrained problem into this framework. By studying the second order geometry of this manifold, we provide a Newton-type algorithm that does not rely on the exponential mapping nor a retraction, while ensuring local convergence guarantees. The algorithm hinges instead upon the developed stability certificate and the linear structure of the constraints. We then apply our methodology to two well-known constrained optimal control problems. Finally, several numerical examples showcase the performance of the proposed algorithm.