Black box approximation in the tensor train format initialized by ANOVA decomposition

TL;DR

This paper introduces a novel initialization method for tensor train surrogate models using ANOVA decomposition, significantly improving accuracy in black box function approximation with limited data.

Contribution

The authors propose using ANOVA-based tensor train initialization for ALS algorithms, enhancing accuracy in high-dimensional surrogate modeling.

Findings

Achieved at least an order of magnitude increase in accuracy.

Demonstrated effectiveness on multidimensional PDE problems.

Applicable to a wide range of surrogate modeling tasks.

Abstract

Surrogate models can reduce computational costs for multivariable functions with an unknown internal structure (black boxes). In a discrete formulation, surrogate modeling is equivalent to restoring a multidimensional array (tensor) from a small part of its elements. The alternating least squares (ALS) algorithm in the tensor train (TT) format is a widely used approach to effectively solve this problem in the case of non-adaptive tensor recovery from a given training set (i.e., tensor completion problem). TT-ALS allows obtaining a low-parametric representation of the tensor, which is free from the curse of dimensionality and can be used for fast computation of the values at arbitrary tensor indices or efficient implementation of algebra operations with the black box (integration, etc.). However, to obtain high accuracy in the presence of restrictions on the size of the train data, a good choice of initial approximation is essential. In this work, we construct the ANOVA representation in the TT-format and use it as an initial approximation for the TT-ALS algorithm. The performed numerical computations for a number of multidimensional model problems, including the parametric partial differential equation, demonstrate a significant advantage of our approach for the commonly used random initial approximation. For all considered model problems we obtained an increase in accuracy by at least an order of magnitude with the same number of requests to the black box. The proposed approach is very general and can be applied in a wide class of real-world surrogate modeling and machine learning problems.