DUST: A Framework for Data-Driven Density Steering

TL;DR

This paper introduces DUST, a novel data-driven framework for covariance steering in stochastic LTI systems, utilizing noisy data to design optimal controllers for steering state distributions.

Contribution

The paper presents a new data-driven approach for covariance steering that characterizes noise and designs controllers directly from data, avoiding reliance on explicit system models.

Findings

DUST effectively estimates noise and bounds errors from data.

The proposed methods outperform model-based approaches in case studies.

Multiple control design approaches are compared and validated.

Abstract

We consider the problem of data-driven stochastic optimal control of an unknown LTI dynamical system. Assuming the process noise is normally distributed, we pose the problem of steering the state's mean and covariance to a target normal distribution, under noisy data collected from the underlying system, a problem commonly referred to as covariance steering (CS). A novel framework for Data-driven Uncertainty quantification and density STeering (DUST) is presented that simultaneously characterizes the noise affecting the measured data and designs an optimal affine-feedback controller to steer the density of the state to a prescribed terminal value. We use both indirect and direct data-driven design approaches based on the notions of persistency of excitation and subspace identification to exactly represent the mean and covariance dynamics of the state in terms of the data and noise realizations. Since both the mean and the covariance steering sub-problems are plagued with stochastic uncertainty arising from noisy data collection, we first estimate the noise realization from this dataset and subsequently compute tractable upper bounds on the estimation errors. The first and second moment steering problems are then solved to optimality using techniques from robust control and robust optimization. Lastly, we present an alternative control design approach based on the certainty equivalence principle and interpret the problem as one of CS under multiplicative uncertainty. We analyze the performance and efficacy of each of these data-driven approaches using a case study and compare them with their model-based counterparts.