Fast Bayesian Updates via Harmonic Representations

TL;DR

This paper presents a fast, spectral convolution-based framework for Bayesian updates using harmonic analysis, significantly improving computational efficiency and enabling real-time probabilistic inference.

Contribution

It introduces a novel spectral approach to Bayesian inference that transforms updates into FFT-based convolutions, reducing complexity and broadening applicability.

Findings

Spectral convolution simplifies Bayesian updates.

FFT accelerates computations to O(N log N).

Method is effective for smooth functions with spectral decay.

Abstract

Bayesian inference, while foundational to probabilistic reasoning, is often hampered by the computational intractability of posterior distributions, particularly through the challenging evidence integral. Conventional approaches like Markov Chain Monte Carlo (MCMC) and Variational Inference (VI) face significant scalability and efficiency limitations. This paper introduces a novel, unifying framework for fast Bayesian updates by leveraging harmonic analysis. We demonstrate that representing the prior and likelihood in a suitable orthogonal basis transforms the Bayesian update rule into a spectral convolution. Specifically, the Fourier coefficients of the posterior are shown to be the normalized convolution of the prior and likelihood coefficients. To achieve computational feasibility, we introduce a spectral truncation scheme, which, for smooth functions, yields an exceptionally accurate finite-dimensional approximation and reduces the update to a circular convolution. This formulation allows us to exploit the Fast Fourier Transform (FFT), resulting in a deterministic algorithm with O(N log N) complexity -- a substantial improvement over the O(N^2) cost of naive methods. We establish rigorous mathematical criteria for the applicability of our method, linking its efficiency to the smoothness and spectral decay of the involved distributions. The presented work offers a paradigm shift, connecting Bayesian computation to signal processing and opening avenues for real-time, sequential inference in a wide class of problems.