On some orthogonalization schemes in Tensor Train format

TL;DR

This paper investigates various orthogonalization methods within the Tensor Train framework, analyzing their orthogonality loss, computational complexity, and stability, with experiments demonstrating the effects of TT-rounding and low-rank approximations.

Contribution

It introduces a detailed analysis of orthogonalization schemes in Tensor Train format, including the impact of TT-rounding on orthogonality and computational efficiency.

Findings

Classical bounds on orthogonality loss are maintained with TT-rounding.

Recompression steps via TT-rounding improve orthogonality preservation.

Computational complexity is dominated by TT-rounding operations.

Abstract

In the framework of tensor spaces, we consider orthogonalization kernels to generate an orthogonal basis of a tensor subspace from a set of linearly independent tensors. In particular, we experimentally study the loss of orthogonality of six orthogonalization methods, namely Classical and Modified Gram-Schmidt with (CGS2, MGS2) and without (CGS, MGS) re-orthogonalization, the Gram approach, and the Householder transformation. To overcome the curse of dimensionality, we represent tensors with a low-rank approximation using the Tensor Train (TT) formalism. In addition, we introduce recompression steps in the standard algorithm outline through the TT-rounding method at a prescribed accuracy. After describing the structure and properties of the algorithms, we illustrate their loss of orthogonality with numerical experiments. The theoretical bounds from the classical matrix computation round-off analysis, obtained over several decades, seem to be maintained, with the unit round-off replaced by the TT-rounding accuracy. The computational analysis for each orthogonalization kernel in terms of the memory requirements and the computational complexity measured as a function of the number of TT-rounding, which happens to be the most computationally expensive operation, completes the study.