Semi-Implicit Stochastic Recurrent Neural Networks

TL;DR

This paper introduces Semi-Implicit Stochastic RNNs that utilize semi-implicit variational inference to learn more flexible implicit latent representations, leading to improved modeling of sequential data.

Contribution

It proposes a novel semi-implicit stochastic RNN framework that enhances latent variable expressiveness beyond Gaussian assumptions using reparameterizable implicit distributions.

Findings

SIS-RNN outperforms existing stochastic RNN methods on real-world datasets.

The model effectively captures complex dependencies in sequential data.

Semi-implicit inference increases model flexibility and performance.

Abstract

Stochastic recurrent neural networks with latent random variables of complex dependency structures have shown to be more successful in modeling sequential data than deterministic deep models. However, the majority of existing methods have limited expressive power due to the Gaussian assumption of latent variables. In this paper, we advocate learning implicit latent representations using semi-implicit variational inference to further increase model flexibility. Semi-implicit stochastic recurrent neural network(SIS-RNN) is developed to enrich inferred model posteriors that may have no analytic density functions, as long as independent random samples can be generated via reparameterization. Extensive experiments in different tasks on real-world datasets show that SIS-RNN outperforms the existing methods.