Efficient Approximation of Volterra Series for High-Dimensional Systems

TL;DR

This paper introduces the Tensor Head Averaging (THA) algorithm, a scalable method that approximates high-dimensional Volterra series models efficiently by combining localized models, backed by theoretical error bounds and analysis.

Contribution

The paper proposes the THA algorithm, which reduces computational complexity in high-dimensional Volterra series identification and provides a theoretical framework for its error analysis.

Findings

THA significantly reduces complexity compared to full MVMALS.

Theoretical bounds on the approximation error are established.

Correlation between included and omitted dynamics enhances accuracy.

Abstract

The identification of high-dimensional nonlinear dynamical systems via the Volterra series has significant potential, but has been severely hindered by the curse of dimensionality. Tensor Network (TN) methods such as the Modified Alternating Linear Scheme (MVMALS) have been a breakthrough for the field, offering a tractable approach by exploiting the low-rank structure in Volterra kernels. However, these techniques still encounter prohibitive computational and memory bottlenecks due to high-order polynomial scaling with respect to input dimension. To overcome this barrier, we introduce the Tensor Head Averaging (THA) algorithm, which significantly reduces complexity by constructing an ensemble of localized MVMALS models trained on small subsets of the input space. In this paper, we present a theoretical foundation for the THA algorithm. We establish observable, finite-sample bounds on the error between the THA ensemble and a full MVMALS model, and we derive an exact decomposition of the squared error. This decomposition is used to analyze the manner in which subset models implicitly compensate for omitted dynamics. We quantify this effect, and prove that correlation between the included and omitted dynamics creates an optimization incentive which drives THA's performance toward accuracy superior to a simple truncation of a full MVMALS model. THA thus offers a scalable and theoretically grounded approach for identifying previously intractable high-dimensional systems.