Hidden State Approximation in Recurrent Neural Networks Using Continuous Particle Filtering

TL;DR

This paper introduces a novel recurrent neural network approach that uses particle filtering to approximate hidden states, enabling better modeling of uncertainty and improved prediction accuracy.

Contribution

It proposes a continuous particle filtering method for RNNs that extends to Encoder-Decoder architectures, enhancing state approximation and adaptability.

Findings

Effective in prediction tasks

Improves hidden state modeling

Demonstrates superior performance

Abstract

Using historical data to predict future events has many applications in the real world, such as stock price prediction; the robot localization. In the past decades, the Convolutional long short-term memory (LSTM) networks have achieved extraordinary success with sequential data in the related field. However, traditional recurrent neural networks (RNNs) keep the hidden states in a deterministic way. In this paper, we use the particles to approximate the distribution of the latent state and show how it can extend into a more complex form, i.e., the Encoder-Decoder mechanism. With the proposed continuous differentiable scheme, our model is capable of adaptively extracting valuable information and updating the latent state according to the Bayes rule. Our empirical studies demonstrate the effectiveness of our method in the prediction tasks.