Learning Stabilizing Policies via an Unstable Subspace Representation

TL;DR

This paper introduces a two-phase method for stabilizing linear systems by learning their unstable subspace, significantly reducing data requirements and speeding up the stabilization process compared to traditional approaches.

Contribution

It proposes a novel two-phase approach that first learns the unstable subspace and then stabilizes it, improving sample efficiency for control of unknown linear systems.

Findings

Learning the unstable subspace reduces sample complexity.

Faster stabilization when unstable modes are few.

Numerical experiments confirm theoretical advantages.

Abstract

We study the problem of learning to stabilize (LTS) a linear time-invariant (LTI) system. Policy gradient (PG) methods for control assume access to an initial stabilizing policy. However, designing such a policy for an unknown system is one of the most fundamental problems in control, and it may be as hard as learning the optimal policy itself. Existing work on the LTS problem requires large data as it scales quadratically with the ambient dimension. We propose a two-phase approach that first learns the left unstable subspace of the system and then solves a series of discounted linear quadratic regulator (LQR) problems on the learned unstable subspace, targeting to stabilize only the system's unstable dynamics and reduce the effective dimension of the control space. We provide non-asymptotic guarantees for both phases and demonstrate that operating on the unstable subspace reduces sample complexity. In particular, when the number of unstable modes is much smaller than the state dimension, our analysis reveals that LTS on the unstable subspace substantially speeds up the stabilization process. Numerical experiments are provided to support this sample complexity reduction achieved by our approach.