Multi-Source Neural Variational Inference

TL;DR

This paper introduces a multi-source variational autoencoder framework that learns shared representations from diverse information sources, addressing redundancy, conflict detection, and robustness in inference.

Contribution

It proposes a novel multi-source learning method with source-conditioned encoders and divergence-based source relation modeling, enhancing inference robustness.

Findings

Effective shared representation learning demonstrated on toy data

Trade-offs identified in separate encoder learning for sources

Conflict detection improves inference robustness

Abstract

Learning from multiple sources of information is an important problem in machine-learning research. The key challenges are learning representations and formulating inference methods that take into account the complementarity and redundancy of various information sources. In this paper we formulate a variational autoencoder based multi-source learning framework in which each encoder is conditioned on a different information source. This allows us to relate the sources via the shared latent variables by computing divergence measures between individual source's posterior approximations. We explore a variety of options to learn these encoders and to integrate the beliefs they compute into a consistent posterior approximation. We visualise learned beliefs on a toy dataset and evaluate our methods for learning shared representations and structured output prediction, showing trade-offs of learning separate encoders for each information source. Furthermore, we demonstrate how conflict detection and redundancy can increase robustness of inference in a multi-source setting.