1-Dimensional polynomial neural networks for audio signal related problems

TL;DR

The paper introduces a 1D Polynomial Neural Network that enhances non-linearity in audio signal processing tasks, achieving better performance with less overall complexity compared to traditional 1DCNNs.

Contribution

It proposes a novel 1D polynomial kernel estimation method for CNNs, increasing non-linearity early in the network to reduce the need for deep or wide architectures.

Findings

Outperforms regular 1DCNNs on multiple audio datasets

Requires less time and memory for training and inference

Extracts more relevant features from data

Abstract

In addition to being extremely non-linear, modern problems require millions if not billions of parameters to solve or at least to get a good approximation of the solution, and neural networks are known to assimilate that complexity by deepening and widening their topology in order to increase the level of non-linearity needed for a better approximation. However, compact topologies are always preferred to deeper ones as they offer the advantage of using less computational units and less parameters. This compacity comes at the price of reduced non-linearity and thus, of limited solution search space. We propose the 1-Dimensional Polynomial Neural Network (1DPNN) model that uses automatic polynomial kernel estimation for 1-Dimensional Convolutional Neural Networks (1DCNNs) and that introduces a high degree of non-linearity from the first layer which can compensate the need for deep and/or wide topologies. We show that this non-linearity enables the model to yield better results with less computational and spatial complexity than a regular 1DCNN on various classification and regression problems related to audio signals, even though it introduces more computational and spatial complexity on a neuronal level. The experiments were conducted on three publicly available datasets and demonstrate that, on the problems that were tackled, the proposed model can extract more relevant information from the data than a 1DCNN in less time and with less memory.