High-Order Tensor Regression in Sparse Convolutional Neural Networks

TL;DR

This paper introduces a tensor-based framework for sparse convolutional neural networks, redefining backpropagation to improve clarity and generality in high-order tensor regression.

Contribution

It develops a rational theory of tensor regression in neural networks and reformulates backpropagation within this new framework.

Findings

A new tensor-based convolution approach is proposed.

Backpropagation is generalized for high-order tensors.

The framework enhances understanding of sparse CNNs.

Abstract

This article presents a generic approach to convolution that significantly differs from conventional methodologies in the current Machine Learning literature. The approach, in its mathematical aspects, proved to be clear and concise, particularly when high-order tensors are involved. In this context, a rational theory of regression in neural networks is developed, as a framework for a generic view of sparse convolutional neural networks, the primary focus of this study. As a direct outcome, the classic Backpropagation Algorithm is redefined to align with this rational tensor-based approach and presented in its simplest, most generic form.