Temporal-Difference Networks for Dynamical Systems with Continuous Observations and Actions

TL;DR

This paper introduces an incremental algorithm for learning predictive models of continuous dynamical systems using temporal-difference networks, extending previous finite-set methods to continuous observations and actions.

Contribution

It presents the first fully incremental method for TD networks applied to continuous dynamical systems, enabling robust modeling of noisy, real-world systems.

Findings

Successfully learned accurate models of noisy continuous systems

Demonstrated robustness of the algorithm in various scenarios

Extended TD networks to continuous observation and action spaces

Abstract

Temporal-difference (TD) networks are a class of predictive state representations that use well-established TD methods to learn models of partially observable dynamical systems. Previous research with TD networks has dealt only with dynamical systems with finite sets of observations and actions. We present an algorithm for learning TD network representations of dynamical systems with continuous observations and actions. Our results show that the algorithm is capable of learning accurate and robust models of several noisy continuous dynamical systems. The algorithm presented here is the first fully incremental method for learning a predictive representation of a continuous dynamical system.