Complex-Valued Neural Networks for Data-Driven Signal Processing and Signal Understanding

TL;DR

This paper introduces a PyTorch-based package for complex-valued neural networks, enabling advanced signal processing and understanding tasks with efficient implementations of core modules and layers.

Contribution

It provides a lightweight, comprehensive toolkit for complex-valued neural network operations, including manifold-based layers, tailored for signal processing and RF signal understanding.

Findings

Implemented efficient complex-valued linear and convolution layers

Developed manifold-based complex neural network modules

Supported 1-D, 2-D, and 3-D data processing

Abstract

Complex-valued neural networks have emerged boasting superior modeling performance for many tasks across the signal processing, sensing, and communications arenas. However, developing complex-valued models currently demands development of basic deep learning operations, such as linear or convolution layers, as modern deep learning frameworks like PyTorch and Tensor flow do not adequately support complex-valued neural networks. This paper overviews a package built on PyTorch with the intention of implementing light-weight interfaces for common complex-valued neural network operations and architectures. Similar to natural language understanding (NLU), which as recently made tremendous leaps towards text-based intelligence, RF Signal Understanding (RFSU) is a promising field extending conventional signal processing algorithms using a hybrid approach of signal mechanics-based insight with data-driven modeling power. Notably, we include efficient implementations for linear, convolution, and attention modules in addition to activation functions and normalization layers such as batchnorm and layernorm. Additionally, we include efficient implementations of manifold-based complex-valued neural network layers that have shown tremendous promise but remain relatively unexplored in many research contexts. Although there is an emphasis on 1-D data tensors, due to a focus on signal processing, communications, and radar data, many of the routines are implemented for 2-D and 3-D data as well. Specifically, the proposed approach offers a useful set of tools and documentation for data-driven signal processing research and practical implementation.