Parameter Efficient Deep Probabilistic Forecasting

TL;DR

This paper introduces a parameter-efficient Bidirectional Temporal Convolutional Network for probabilistic time series forecasting, achieving comparable accuracy to state-of-the-art models while significantly reducing memory and training costs.

Contribution

The novel BiTCN model combines two TCNs for encoding past and future covariates, requiring fewer parameters than Transformer-based models, with comparable forecasting performance.

Findings

Performs on par with state-of-the-art methods on real datasets

Uses significantly fewer parameters, reducing training time and memory

Maintains accuracy across multiple evaluation metrics

Abstract

Probabilistic time series forecasting is crucial in many application domains such as retail, ecommerce, finance, or biology. With the increasing availability of large volumes of data, a number of neural architectures have been proposed for this problem. In particular, Transformer-based methods achieve state-of-the-art performance on real-world benchmarks. However, these methods require a large number of parameters to be learned, which imposes high memory requirements on the computational resources for training such models. To address this problem, we introduce a novel Bidirectional Temporal Convolutional Network (BiTCN), which requires an order of magnitude less parameters than a common Transformer-based approach. Our model combines two Temporal Convolutional Networks (TCNs): the first network encodes future covariates of the time series, whereas the second network encodes past observations and covariates. We jointly estimate the parameters of an output distribution via these two networks. Experiments on four real-world datasets show that our method performs on par with four state-of-the-art probabilistic forecasting methods, including a Transformer-based approach and WaveNet, on two point metrics (sMAPE, NRMSE) as well as on a set of range metrics (quantile loss percentiles) in the majority of cases. Secondly, we demonstrate that our method requires significantly less parameters than Transformer-based methods, which means the model can be trained faster with significantly lower memory requirements, which as a consequence reduces the infrastructure cost for deploying these models.