Pyramidal Predictive Network: A Model for Visual-frame Prediction Based on Predictive Coding Theory

TL;DR

This paper introduces a pyramidal neural network inspired by predictive coding theory for visual-frame prediction, achieving better efficiency and comparable accuracy with lower computational cost.

Contribution

The paper proposes a novel pyramidal predictive network model that integrates predictive coding and multi-frequency oscillations for improved visual-frame prediction.

Findings

Better compactness and efficiency compared to existing models

Achieves comparable predictive accuracy

Lower computational cost

Abstract

Visual-frame prediction is a pixel-dense prediction task that infers future frames from past frames. Lacking of appearance details, low prediction accuracy and high computational overhead are still major problems with current models or methods. In this paper, we propose a novel neural network model inspired by the well-known predictive coding theory to deal with the problems. Predictive coding provides an interesting and reliable computational framework, which will be combined with other theories such as the cerebral cortex at different level oscillates at different frequencies, to design an efficient and reliable predictive network model for visual-frame prediction. Specifically, the model is composed of a series of recurrent and convolutional units forming the top-down and bottom-up streams, respectively. The update frequency of neural units on each of the layer decreases with the increasing of network levels, which results in neurons of higher-level can capture information in longer time dimensions. According to the experimental results, this model shows better compactness and comparable predictive performance with existing works, implying lower computational cost and higher prediction accuracy. Code is available at https://github.com/Ling-CF/PPNet.