State Estimation and Control of Dynamic Systems from High-Dimensional Image Data

TL;DR

This paper presents a neural network architecture combining CNNs and GRUs to estimate states from image sequences for reinforcement learning, enabling real-time control without ground-truth states.

Contribution

It introduces a novel neural architecture that effectively learns state representations from high-dimensional image data for control tasks.

Findings

Enables real-time, accurate state estimation from images.

Improves policy performance without access to true states.

Provides a new evaluation methodology for learned state accuracy.

Abstract

Accurate state estimation is critical for optimal policy design in dynamic systems. However, obtaining true system states is often impractical or infeasible, complicating the policy learning process. This paper introduces a novel neural architecture that integrates spatial feature extraction using convolutional neural networks (CNNs) and temporal modeling through gated recurrent units (GRUs), enabling effective state representation from sequences of images and corresponding actions. These learned state representations are used to train a reinforcement learning agent with a Deep Q-Network (DQN). Experimental results demonstrate that our proposed approach enables real-time, accurate estimation and control without direct access to ground-truth states. Additionally, we provide a quantitative evaluation methodology for assessing the accuracy of the learned states, highlighting their impact on policy performance and control stability.