Fully complex-valued deep learning model for visual perception

TL;DR

This paper introduces a fully complex-valued convolutional neural network that operates entirely in the complex domain, leading to improved performance, efficiency, and convergence compared to real-valued and other complex models.

Contribution

It proposes a novel fully complex-valued training scheme and loss function, achieving state-of-the-art results with fewer parameters and faster training on standard image datasets.

Findings

4-10% accuracy improvement over real-valued models

Fewer parameters needed for comparable performance

Faster convergence and better training efficiency

Abstract

Deep learning models operating in the complex domain are used due to their rich representation capacity. However, most of these models are either restricted to the first quadrant of the complex plane or project the complex-valued data into the real domain, causing a loss of information. This paper proposes that operating entirely in the complex domain increases the overall performance of complex-valued models. A novel, fully complex-valued learning scheme is proposed to train a Fully Complex-valued Convolutional Neural Network (FC-CNN) using a newly proposed complex-valued loss function and training strategy. Benchmarked on CIFAR-10, SVHN, and CIFAR-100, FC-CNN has a 4-10% gain compared to its real-valued counterpart, maintaining the model complexity. With fewer parameters, it achieves comparable performance to state-of-the-art complex-valued models on CIFAR-10 and SVHN. For the CIFAR-100 dataset, it achieves state-of-the-art performance with 25% fewer parameters. FC-CNN shows better training efficiency and much faster convergence than all the other models.