Complex Gated Recurrent Neural Networks

TL;DR

This paper introduces a novel complex gated recurrent neural network cell that combines complex-valued and norm-preserving state transitions, enhancing stability and performance in sequence processing tasks.

Contribution

The paper proposes a new complex gated recurrent cell that improves stability and convergence in RNNs by integrating complex-valued and norm-preserving mechanisms.

Findings

Performs well on synthetic memory and adding tasks

Achieves competitive results in human motion prediction

Exhibits excellent stability and convergence properties

Abstract

Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures. RNNs, widely used to process time series and sequence information, could greatly benefit from complex representations. We present a novel complex gated recurrent cell, which is a hybrid cell combining complex-valued and norm-preserving state transitions with a gating mechanism. The resulting RNN exhibits excellent stability and convergence properties and performs competitively on the synthetic memory and adding task, as well as on the real-world tasks of human motion prediction.