An Efficient Approach to Regression Problems with Tensor Neural Networks

TL;DR

This paper presents a tensor neural network (TNN) that improves high-dimensional regression by effectively separating variables, integrating statistical and numerical methods, and analyzing gradients to identify key data features.

Contribution

The paper introduces a novel tensor neural network architecture that enhances approximation and generalization in high-dimensional regression tasks, integrating statistical and numerical techniques.

Findings

TNN outperforms FFN and RBN in accuracy and generalization.

Efficient high-dimensional integral computation within TNN.

Gradient and Laplacian analysis identifies key predictive dimensions.

Abstract

This paper introduces a tensor neural network (TNN) to address nonparametric regression problems, leveraging its distinct sub-network structure to effectively facilitate variable separation and enhance the approximation of complex, high-dimensional functions. The TNN demonstrates superior performance compared to conventional Feed-Forward Networks (FFN) and Radial Basis Function Networks (RBN) in terms of both approximation accuracy and generalization capacity, even with a comparable number of parameters. A significant innovation in our approach is the integration of statistical regression and numerical integration within the TNN framework. This allows for efficient computation of high-dimensional integrals associated with the regression function and provides detailed insights into the underlying data structure. Furthermore, we employ gradient and Laplacian analysis on the regression outputs to identify key dimensions influencing the predictions, thereby guiding the design of subsequent experiments. These advancements make TNN a powerful tool for applications requiring precise high-dimensional data analysis and predictive modeling.