Suboptimal coverings for continuous spaces of control tasks

TL;DR

This paper introduces the {}-suboptimal covering number to quantify the complexity of multi-task control problems with infinite task sets, aiding in understanding the function class expressiveness needed for learning-based control.

Contribution

It defines a new measure called the {}-suboptimal covering number for continuous control spaces and analyzes its properties for linear quadratic regulator (LQR) problems.

Findings

Logarithmic dependence on the space 'breadth' for scalar cases.

Constructive covers can efficiently approximate multi-task LQR problems.

Experiments visualize lower bounds and system behaviors for the proposed covering measure.

Abstract

We propose the {\alpha}-suboptimal covering number to characterize multi-task control problems where the set of dynamical systems and/or cost functions is infinite, analogous to the cardinality of finite task sets. This notion may help quantify the function class expressiveness needed to represent a good multi-task policy, which is important for learning-based control methods that use parameterized function approximation. We study suboptimal covering numbers for linear dynamical systems with quadratic cost (LQR problems) and construct a class of multi-task LQR problems amenable to analysis. For the scalar case, we show logarithmic dependence on the "breadth" of the space. For the matrix case, we present experiments 1) measuring the efficiency of a particular constructive cover, and 2) visualizing the behavior of two candidate systems for the lower bound.