Predictive Coding for Dynamic Visual Processing: Development of Functional Hierarchy in a Multiple Spatio-Temporal Scales RNN Model

TL;DR

This paper introduces P-MSTRNN, a neural network model based on predictive coding that learns to predict and imitate human cyclic movements by developing a hierarchical dynamic structure across multiple spatio-temporal scales.

Contribution

The paper presents a novel predictive coding neural network with multiscale spatio-temporal constraints, demonstrating hierarchical dynamic development and early transient dynamics for movement prediction and imitation.

Findings

Model develops a functional hierarchy with layer-specific dynamics.

Number of attractors increases with learning progress.

Transient dynamics enable early-stage pattern generation.

Abstract

The current paper proposes a novel predictive coding type neural network model, the predictive multiple spatio-temporal scales recurrent neural network (P-MSTRNN). The P-MSTRNN learns to predict visually perceived human whole-body cyclic movement patterns by exploiting multiscale spatio-temporal constraints imposed on network dynamics by using differently sized receptive fields as well as different time constant values for each layer. After learning, the network becomes able to proactively imitate target movement patterns by inferring or recognizing corresponding intentions by means of the regression of prediction error. Results show that the network can develop a functional hierarchy by developing a different type of dynamic structure at each layer. The paper examines how model performance during pattern generation as well as predictive imitation varies depending on the stage of learning. The number of limit cycle attractors corresponding to target movement patterns increases as learning proceeds. And, transient dynamics developing early in the learning process successfully perform pattern generation and predictive imitation tasks. The paper concludes that exploitation of transient dynamics facilitates successful task performance during early learning periods.