Reinforcement Learning Approach to Estimation in Linear Systems

TL;DR

This paper introduces a reinforcement learning framework for system identification and state estimation in linear systems, specifically ARMAX models, and applies these methods to model-free LQG control.

Contribution

It presents a new RL-based algorithm for consistent system identification and a novel approach to model-free state estimation in linear systems.

Findings

Proposed RL algorithm guarantees consistency in system identification.

Developed a new method for model-free state estimation.

Applied methods successfully to model-free LQG control problem.

Abstract

This paper addresses two important estimation problems for linear systems, namely system identification and model-free state estimation. Our focus is on ARMAX models with unknown parameters. We first provide a reinforcement learning algorithm for system identification with guaranteed consistency. This algorithm is then used to provide a novel solution to model-free state estimation. These results are then applied to solving the model-free LQG control problem in the reinforcement learning setting.