An analysis of deep neural networks for predicting trends in time series data

TL;DR

This paper re-evaluates the TreNet hybrid deep learning model for time series trend prediction using a more appropriate validation method, revealing its performance advantages are dataset-dependent and emphasizing the importance of validation techniques.

Contribution

It demonstrates the significance of proper validation methods and stability testing in assessing deep learning models for time series trend prediction.

Findings

TreNet generally outperforms vanilla DNN models

Model performance varies across datasets

Validation method impacts performance assessment

Abstract

Recently, a hybrid Deep Neural Network (DNN) algorithm, TreNet was proposed for predicting trends in time series data. While TreNet was shown to have superior performance for trend prediction to other DNN and traditional ML approaches, the validation method used did not take into account the sequential nature of time series data sets and did not deal with model update. In this research we replicated the TreNet experiments on the same data sets using a walk-forward validation method and tested our optimal model over multiple independent runs to evaluate model stability. We compared the performance of the hybrid TreNet algorithm, on four data sets to vanilla DNN algorithms that take in point data, and also to traditional ML algorithms. We found that in general TreNet still performs better than the vanilla DNN models, but not on all data sets as reported in the original TreNet study. This study highlights the importance of using an appropriate validation method and evaluating model stability for evaluating and developing machine learning models for trend prediction in time series data.