Constructing structured tensor priors for Bayesian inverse problems

TL;DR

This paper characterizes Gaussian priors for structured tensor solutions in Bayesian inverse problems, enabling new kernel design and application to matrix completion and image classification.

Contribution

It introduces a complete characterization of Gaussian priors for structured tensors, including explicit covariance expressions, facilitating new prior design for inverse problems.

Findings

Effective tensor priors for Hankel matrix completion.

Successful application to handwritten digit classification.

Demonstrated computational efficiency with Julia notebooks.

Abstract

Specifying a prior distribution is an essential part of solving Bayesian inverse problems. The prior encodes a belief on the nature of the solution and this regularizes the problem. In this article we completely characterize a Gaussian prior that encodes the belief that the solution is a structured tensor. We first define the notion of (A,b)-constrained tensors and show that they describe a large variety of different structures such as Hankel, circulant, triangular, symmetric, and so on. Then we completely characterize the Gaussian probability distribution of such tensors by specifying its mean vector and covariance matrix. Furthermore, explicit expressions are proved for the covariance matrix of tensors whose entries are invariant under a permutation. These results unlock a whole new class of priors for Bayesian inverse problems. We illustrate how new kernel functions can be designed and efficiently computed and apply our results on two particular Bayesian inverse problems: completing a Hankel matrix from a few noisy measurements and learning an image classifier of handwritten digits. The effectiveness of the proposed priors is demonstrated for both problems. All applications have been implemented as reactive Pluto notebooks in Julia.