Differential radial basis function network for sequence modelling

TL;DR

The paper introduces RBF-DiffNet, a novel differential RBF network that models sequences via PDEs, improving noise robustness and prediction accuracy in time series forecasting compared to traditional RBF, ARIMA, and LSTM models.

Contribution

It proposes a differential RBF network with PDE-based hidden layers, enhancing noise robustness and computational efficiency in sequence modeling.

Findings

RBF-DiffNet reduces prediction error by 26% on M5 dataset.

It performs comparably to LSTM ensembles with significantly less computation.

The model improves robustness to observational noise in time series forecasting.

Abstract

We propose a differential radial basis function (RBF) network termed RBF-DiffNet -- whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF -- to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction over the baseline RBF network in terms of the prediction error (e.g., 26% reduction in the root mean squared scaled error on the M5 dataset); RBF-DiffNet also shows a comparable performance to the LSTM ensemble at less than one-sixteenth the LSTM computational time. Our proposed network consequently enables more accurate predictions -- in the presence of observational noise -- in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.