Complex Valued Gated Auto-encoder for Video Frame Prediction

TL;DR

This paper introduces a complex valued gated auto-encoder for video frame prediction, leveraging complex neural networks to learn transformations efficiently, with improved performance and parameter efficiency demonstrated on synthetic datasets.

Contribution

It presents a novel complex valued neural network architecture for video prediction, showing advantages over real-valued models and the benefits of convolutional extensions.

Findings

Complex neural networks can effectively learn translational transformations.

Complex models outperform real-valued counterparts in prediction accuracy.

Convolutional extensions improve parameter efficiency and performance.

Abstract

In recent years, complex valued artificial neural networks have gained increasing interest as they allow neural networks to learn richer representations while potentially incorporating less parameters. Especially in the domain of computer graphics, many traditional operations rely heavily on computations in the complex domain, thus complex valued neural networks apply naturally. In this paper, we perform frame predictions in video sequences using a complex valued gated auto-encoder. First, our method is motivated showing how the Fourier transform can be seen as the basis for translational operations. Then, we present how a complex neural network can learn such transformations and compare its performance and parameter efficiency to a real-valued gated autoencoder. Furthermore, we show how extending both - the real and the complex valued - neural networks by using convolutional units can significantly improve prediction performance and parameter efficiency. The networks are assessed on a moving noise and a bouncing ball dataset.