Functional Complexity-adaptive Temporal Tensor Decomposition

TL;DR

This paper introduces extsc{Catte}, a novel functional tensor decomposition method that adaptively models continuous spatial and temporal data using neural ODEs and Fourier features, improving prediction accuracy and robustness.

Contribution

It proposes a new functional temporal tensor decomposition framework with adaptive complexity, leveraging neural ODEs and Fourier features, and introduces a sparsity prior for automatic model complexity adjustment.

Findings

Outperforms existing methods in prediction accuracy.

Effectively reveals underlying tensor ranks.

Demonstrates robustness against noise.

Abstract

Tensor decomposition is a fundamental tool for analyzing multi-dimensional data by learning low-rank factors to represent high-order interactions. While recent works on temporal tensor decomposition have made significant progress by incorporating continuous timestamps in latent factors, they still struggle with general tensor data with continuous indexes not only in the temporal mode but also in other modes, such as spatial coordinates in climate data. Moreover, the challenge of self-adapting model complexity is largely unexplored in functional temporal tensor models, with existing methods being inapplicable in this setting. To address these limitations, we propose functional \underline{C}omplexity-\underline{A}daptive \underline{T}emporal \underline{T}ensor d\underline{E}composition (\textsc{Catte}). Our approach encodes continuous spatial indexes as learnable Fourier features and employs neural ODEs in latent space to learn the temporal trajectories of factors. To enable automatic adaptation of model complexity, we introduce a sparsity-inducing prior over the factor trajectories. We develop an efficient variational inference scheme with an analytical evidence lower bound, enabling sampling-free optimization. Through extensive experiments on both synthetic and real-world datasets, we demonstrate that \textsc{Catte} not only reveals the underlying ranks of functional temporal tensors but also significantly outperforms existing methods in prediction performance and robustness against noise.