Neural Networks with Complex-Valued Weights Have No Spurious Local Minima

TL;DR

This paper proves that shallow neural networks with complex-valued weights and quadratic activations have no spurious local minima, unlike their real-valued counterparts, highlighting the advantages of complex weights in optimization landscapes.

Contribution

It establishes the absence of spurious local minima in shallow complex neural networks with quadratic activations, contrasting with real networks.

Findings

Complex neural networks lack spurious local minima.

Real neural networks have infinitely many spurious minima.

Complex weights can convert poor minima into saddle points.

Abstract

We study the benefits of complex-valued weights for neural networks. We prove that shallow complex neural networks with quadratic activations have no spurious local minima. In contrast, shallow real neural networks with quadratic activations have infinitely many spurious local minima under the same conditions. In addition, we provide specific examples to demonstrate that complex-valued weights turn poor local minima into saddle points.