Backpropagation with N-D Vector-Valued Neurons Using Arbitrary Bilinear Products

TL;DR

This paper introduces ABIPNN, a novel vector neural network architecture utilizing arbitrary bilinear products, which enhances learning by modeling associations among vector components, demonstrated through applications in image denoising and voice separation.

Contribution

The paper proposes ABIPNN, a new neural network architecture that processes vectors with arbitrary bilinear products, capturing associations among vector components for improved performance.

Findings

ABIPNN outperforms conventional neural networks in experiments.

Associations among vector components are effectively learned.

Applications include multispectral image denoising and voice separation.

Abstract

Vector-valued neural learning has emerged as a promising direction in deep learning recently. Traditionally, training data for neural networks (NNs) are formulated as a vector of scalars; however, its performance may not be optimal since associations among adjacent scalars are not modeled. In this paper, we propose a new vector neural architecture called the Arbitrary BIlinear Product Neural Network (ABIPNN), which processes information as vectors in each neuron, and the feedforward projections are defined using arbitrary bilinear products. Such bilinear products can include circular convolution, seven-dimensional vector product, skew circular convolution, reversed- time circular convolution, or other new products not seen in previous work. As a proof-of-concept, we apply our proposed network to multispectral image denoising and singing voice sepa- ration. Experimental results show that ABIPNN gains substantial improvements when compared to conventional NNs, suggesting that associations are learned during training.