Encoding Longer-term Contextual Multi-modal Information in a Predictive Coding Model

TL;DR

This paper introduces a hierarchical predictive coding neural network model that captures multi-scale temporal predictions and motor modulation, tested on neurorobotic platforms to understand long-term contextual multi-modal information processing.

Contribution

It proposes a novel hierarchical predictive coding model with multi-scale temporal dynamics and motor modulation, validated through neurorobotic experiments.

Findings

Effective modeling of long-term contextual information.

Successful implementation on neurorobotic platform.

Demonstrated integration of motor activities with hierarchical predictions.

Abstract

Studies suggest that within the hierarchical architecture, the topological higher level possibly represents a conscious category of the current sensory events with slower changing activities. They attempt to predict the activities on the lower level by relaying the predicted information. On the other hand, the incoming sensory information corrects such prediction of the events on the higher level by the novel or surprising signal. We propose a predictive hierarchical artificial neural network model that examines this hypothesis on neurorobotic platforms, based on the AFA-PredNet model. In this neural network model, there are different temporal scales of predictions exist on different levels of the hierarchical predictive coding, which are defined in the temporal parameters in the neurons. Also, both the fast and the slow-changing neural activities are modulated by the active motor activities. A neurorobotic experiment based on the architecture was also conducted based on the data collected from the VRep simulator.