The effects of regularisation on RNN models for time series forecasting: Covid-19 as an example

TL;DR

This paper investigates how regularisation techniques, especially Dropout, improve RNN models like GRU for time series forecasting on small datasets, using COVID-19 data as a case study.

Contribution

It introduces a flexible RNN model optimized for small datasets and systematically evaluates regularisation methods to enhance forecasting accuracy.

Findings

Dropout with 20% rate yields lowest RMSE in experiments.

Regularisation is more crucial for models with limited data access.

Applying Dropout on 28 days of COVID-19 data reduces RMSE by 23%.

Abstract

Many research papers that propose models to predict the course of the COVID-19 pandemic either use handcrafted statistical models or large neural networks. Even though large neural networks are more powerful than simpler statistical models, they are especially hard to train on small datasets. This paper not only presents a model with grater flexibility than the other proposed neural networks, but also presents a model that is effective on smaller datasets. To improve performance on small data, six regularisation methods were tested. The results show that the GRU combined with 20% Dropout achieved the lowest RMSE scores. The main finding was that models with less access to data relied more on the regulariser. Applying Dropout to a GRU model trained on only 28 days of data reduced the RMSE by 23%.