Tensor-Based Foundations of Ordinary Least Squares and Neural Network Regression Models

TL;DR

This paper develops a tensor-based mathematical framework for Ordinary Least Squares and Neural Network regression models, providing new theoretical insights and algorithms, including an improved backpropagation method.

Contribution

It introduces a novel tensor analysis approach to foundational regression models, extending their theory and deriving new algorithms for neural networks.

Findings

New tensor-based theoretical foundations for OLS and neural networks

Development of three algorithms, including an improved backpropagation

Enhanced understanding of model computations through tensor analysis

Abstract

This article introduces a novel approach to the mathematical development of Ordinary Least Squares and Neural Network regression models, diverging from traditional methods in current Machine Learning literature. By leveraging Tensor Analysis and fundamental matrix computations, the theoretical foundations of both models are meticulously detailed and extended to their complete algorithmic forms. The study culminates in the presentation of three algorithms, including a streamlined version of the Backpropagation Algorithm for Neural Networks, illustrating the benefits of this new mathematical approach.