The M-Tensor Format: Optimality in High Dimensional Regression for Nonlinear Models with Scarce Data

TL;DR

This paper introduces the M-Tensor format, a novel nonlinear regression approach using tensor algebra that effectively handles high-dimensional, scarce data scenarios by leveraging structured equations and regularization techniques.

Contribution

It develops a new tensor-based regression framework that combines kernel methods and tensor algebra to address high-dimensional, low-data problems without fixed point iterations.

Findings

Demonstrates robustness on engineering benchmarks and dynamical systems.

Shows scalability in complexity with increasing parameters.

Provides an implementation that avoids the curse of dimensionality.

Abstract

We present a nonlinear regression framework based on tensor algebra tailored to high dimensional contexts where data is scarce. We exploit algebraic properties of a partial tensor product, namely the m-tensor product, to leverage structured equations with separated variables. The proposed method combines kernel properties along with tensor algebra to prevent the curse of dimensionality and tackle approximations up to hundreds of parameters while avoiding the fixed point strategy. This formalism allows us to provide different regularization techniques fit for low amount of data with a high number of parameters while preserving well-known matrix-based properties. We demonstrate complexity scaling on a general benchmark and dynamical systems to show robustness for engineering problems and ease of implementation.