Learning to Control using Image Feedback

TL;DR

This paper introduces a neural network-based control framework that uses image feedback to learn control policies for complex systems, employing deep Q-networks and a novel error-driven learning method.

Contribution

It presents a new control strategy using deep Q-networks with image feedback and a direct error-driven learning approach for training neural networks.

Findings

Effective control policy synthesis demonstrated through numerical examples.

The framework successfully handles image-based feedback in control tasks.

The proposed learning method improves training efficiency and accuracy.

Abstract

Learning to control complex systems using non-traditional feedback, e.g., in the form of snapshot images, is an important task encountered in diverse domains such as robotics, neuroscience, and biology (cellular systems). In this paper, we present a two neural-network (NN)-based feedback control framework to design control policies for systems that generate feedback in the form of images. In particular, we develop a deep $Q$-network (DQN)-driven learning control strategy to synthesize a sequence of control inputs from snapshot images that encode the information pertaining to the current state and control action of the system. Further, to train the networks we employ a direct error-driven learning (EDL) approach that utilizes a set of linear transformations of the NN training error to update the NN weights in each layer. We verify the efficacy of the proposed control strategy using numerical examples.