A Tensor Train Approach for Deterministic Arithmetic Operations on Discrete Representations of Probability Distributions

TL;DR

This paper introduces a tensor train method for exact, deterministic arithmetic on high-dimensional probability distributions, significantly reducing memory and computational costs compared to traditional stochastic or exponential approaches.

Contribution

It develops a low-rank tensor train framework for efficient, exact arithmetic on discretized probability distributions, avoiding the curse of dimensionality.

Findings

Orders-of-magnitude memory reduction

Significant computational speedups

Polynomial complexity under low-rank assumptions

Abstract

Computing with discrete representations of high-dimensional probability distributions is fundamental to uncertainty quantification, Bayesian inference, and stochastic modeling. However, storing and manipulating such distributions suffers from the curse of dimensionality, as memory and computational costs grow exponentially with dimension. Monte Carlo methods require thousands to billions of samples, incurring high computational costs and producing inconsistent results due to stochasticity. We present an efficient tensor train method for performing exact arithmetic operations on discretizations of continuous probability distributions while avoiding exponential growth. Our approach leverages low-rank tensor train decomposition to represent latent random variables compactly using Dirac deltas, enabling deterministic addition, subtraction and multiplication operations directly in the compressed format. We develop an efficient implementation using sparse matrices and specialized data structures that further enhances performance. Theoretical analysis demonstrates polynomial scaling of memory and computational complexity under rank assumptions, and shows how statistics of latent variables can be computed with polynomial complexity. Numerical experiments spanning randomized linear algebra to stochastic differential equations demonstrate orders-of-magnitude improvements in memory usage and computational time compared to conventional approaches, enabling tractable deterministic computations on discretized random variables in previously intractable dimensions.