Exploring multimodal data fusion through joint decompositions with flexible couplings

TL;DR

This paper introduces a Bayesian framework for flexible joint tensor decompositions to improve multimodal data fusion, providing algorithms, theoretical bounds, and simulations for various coupling models.

Contribution

It develops a Bayesian approach with flexible coupling models for joint tensor decompositions, including algorithms and performance bounds for large and diverse datasets.

Findings

Bayesian framework enables flexible data fusion models.

Algorithms for joint MAP estimation are discussed.

Performance bounds are derived for Gaussian couplings.

Abstract

A Bayesian framework is proposed to define flexible coupling models for joint tensor decompositions of multiple data sets. Under this framework, a natural formulation of the data fusion problem is to cast it in terms of a joint maximum a posteriori (MAP) estimator. Data driven scenarios of joint posterior distributions are provided, including general Gaussian priors and non Gaussian coupling priors. We present and discuss implementation issues of algorithms used to obtain the joint MAP estimator. We also show how this framework can be adapted to tackle the problem of joint decompositions of large datasets. In the case of a conditional Gaussian coupling with a linear transformation, we give theoretical bounds on the data fusion performance using the Bayesian Cramer-Rao bound. Simulations are reported for hybrid coupling models ranging from simple additive Gaussian models, to Gamma-type models with positive variables and to the coupling of data sets which are inherently of different size due to different resolution of the measurement devices.