Particle Filter Recurrent Neural Networks

TL;DR

This paper introduces Particle Filter Recurrent Neural Networks (PF-RNNs), which explicitly model uncertainty in sequential data prediction by maintaining a distribution of latent states, leading to improved performance over standard RNNs.

Contribution

The paper proposes a novel RNN architecture that incorporates particle filtering to explicitly model uncertainty, with a fully differentiable algorithm for training.

Findings

PF-RNNs outperform standard gated RNNs on synthetic and real-world datasets.

The approach effectively handles noisy and variable data.

Experimental results show improved prediction accuracy.

Abstract

Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and noisy real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle filter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classification, stock price prediction, etc.