Interpretable Deep Representation Learning from Temporal Multi-view Data

TL;DR

This paper introduces a generative model combining variational autoencoders and recurrent neural networks to analyze multi-view temporal data, enabling disentangled and interpretable latent representations across diverse scientific domains.

Contribution

The paper presents a novel generative model that integrates multi-view and temporal data for improved interpretability and disentanglement of latent factors.

Findings

Effective analysis of three diverse datasets

Enhanced interpretability of latent representations

Successful disentanglement of multi-view temporal factors

Abstract

In many scientific problems such as video surveillance, modern genomics, and finance, data are often collected from diverse measurements across time that exhibit time-dependent heterogeneous properties. Thus, it is important to not only integrate data from multiple sources (called multi-view data), but also to incorporate time dependency for deep understanding of the underlying system. We propose a generative model based on variational autoencoder and a recurrent neural network to infer the latent dynamics for multi-view temporal data. This approach allows us to identify the disentangled latent embeddings across views while accounting for the time factor. We invoke our proposed model for analyzing three datasets on which we demonstrate the effectiveness and the interpretability of the model.