Towards Recurrent Autoregressive Flow Models

TL;DR

This paper introduces Recurrent Autoregressive Flows, a novel approach combining normalizing flows and recurrent neural networks to model complex non-stationary stochastic processes.

Contribution

It proposes a new recurrent flow model architecture and training method for better representation of non-stationary stochastic processes.

Findings

Effective modeling of complex stochastic processes demonstrated

Identified limitations of current recurrent flow design

Potential solutions for improving model performance suggested

Abstract

Stochastic processes generated by non-stationary distributions are difficult to represent with conventional models such as Gaussian processes. This work presents Recurrent Autoregressive Flows as a method toward general stochastic process modeling with normalizing flows. The proposed method defines a conditional distribution for each variable in a sequential process by conditioning the parameters of a normalizing flow with recurrent neural connections. Complex conditional relationships are learned through the recurrent network parameters. In this work, we present an initial design for a recurrent flow cell and a method to train the model to match observed empirical distributions. We demonstrate the effectiveness of this class of models through a series of experiments in which models are trained on three complex stochastic processes. We highlight the shortcomings of our current formulation and suggest some potential solutions.