Direct Method for Training Feed-forward Neural Networks using Batch Extended Kalman Filter for Multi-Step-Ahead Predictions

TL;DR

This paper introduces a novel batch training method for feed-forward neural networks using an extended Kalman filter, specifically designed for multi-step-ahead time series prediction, demonstrated on chaotic and laser data series.

Contribution

It presents a new batch derivative calculation technique and modifies the Extended Kalman Filter for improved training of feed-forward networks in multi-step predictions.

Findings

Effective on chaotic Mackey-Glass data

Successful on Santa Fe Laser Data Series

Outperforms traditional training methods

Abstract

This paper is dedicated to the long-term, or multi-step-ahead, time series prediction problem. We propose a novel method for training feed-forward neural networks, such as multilayer perceptrons, with tapped delay lines. Special batch calculation of derivatives called Forecasted Propagation Through Time and batch modification of the Extended Kalman Filter are introduced. Experiments were carried out on well-known time series benchmarks, the Mackey-Glass chaotic process and the Santa Fe Laser Data Series. Recurrent and feed-forward neural networks were evaluated.