It's a super deal -- train recurrent network on noisy data and get smooth prediction free

TL;DR

Training recurrent neural networks on noisy data can lead to smooth and accurate future predictions, with noise compression playing a key role, relevant for both machine learning and neuroscience.

Contribution

This paper analyzes how noise in training and input data affects recurrent network predictions and explains the noise compression phenomenon observed.

Findings

Noise compression occurs during prediction, smoothing the output trajectories.

Training on noisy data improves the smoothness and accuracy of predictions.

The noise compression property has implications for understanding neural processing in biology.

Abstract

Recent research demonstrate that prediction of time series by predictive recurrent neural networks based on the noisy input generates a smooth anticipated trajectory. We examine influence of the noise component in both the training data sets and the input sequences on network prediction quality. We propose and discuss an explanation of the observed noise compression in the predictive process. We also discuss importance of this property of recurrent networks in the neuroscience context for the evolution of living organisms.