A block-sparse Tensor Train Format for sample-efficient high-dimensional Polynomial Regression

TL;DR

This paper introduces a block-sparse Tensor Train format for high-dimensional polynomial regression, improving sample efficiency and computational resource use by leveraging structured sparsity patterns aligned with polynomial subspaces.

Contribution

It extends low-rank tensor methods by incorporating block-sparsity, enabling better adaptation to polynomial subspaces and enhancing efficiency in high-dimensional regression.

Findings

Demonstrates improved sample efficiency in numerical experiments

Shows better computational resource utilization

Aligns sparsity patterns with polynomial subspaces

Abstract

Low-rank tensors are an established framework for high-dimensional least-squares problems. We propose to extend this framework by including the concept of block-sparsity. In the context of polynomial regression each sparsity pattern corresponds to some subspace of homogeneous multivariate polynomials. This allows us to adapt the ansatz space to align better with known sample complexity results. The resulting method is tested in numerical experiments and demonstrates improved computational resource utilization and sample efficiency.