Laplace Approximation for Bayesian Tensor Network Kernel Machines

TL;DR

This paper introduces LA-TNKM, a Bayesian tensor network kernel machine that uses Laplace approximation for uncertainty estimation, outperforming GPs and BNNs on various regression benchmarks.

Contribution

It presents a novel Bayesian inference method for tensor network kernel machines using Laplace approximation, enabling principled uncertainty quantification.

Findings

LA-TNKM matches or surpasses GPs and BNNs in regression tasks.

The method provides effective uncertainty estimates in diverse benchmarks.

Tensor network assumptions can be integrated with Bayesian inference for scalable models.

Abstract

Uncertainty estimation is essential for robust decision-making in the presence of ambiguous or out-of-distribution inputs. Gaussian Processes (GPs) are classical kernel-based models that offer principled uncertainty quantification and perform well on small- to medium-scale datasets. Alternatively, formulating the weight space learning problem under tensor network assumptions yields scalable tensor network kernel machines. However, these assumptions break Gaussianity, complicating standard probabilistic inference. This raises a fundamental question: how can tensor network kernel machines provide principled uncertainty estimates? We propose a novel Bayesian Tensor Network Kernel Machine (LA-TNKM) that employs a (linearized) Laplace approximation for Bayesian inference. A comprehensive set of numerical experiments shows that the proposed method consistently matches or surpasses Gaussian Processes and Bayesian Neural Networks (BNNs) across diverse UCI regression benchmarks, highlighting both its effectiveness and practical relevance.