Model Compression for Dynamic Forecast Combination

TL;DR

This paper explores compressing dynamic ensemble models in time series forecasting to reduce computational costs and improve interpretability while maintaining predictive accuracy.

Contribution

It introduces the novel application of model compression to various forecasting methods, demonstrating effectiveness in dynamic, non-stationary time series data.

Findings

Compressed models achieve similar accuracy to ensembles.

Significant reduction in computational costs.

Rule-based models offer best interpretability.

Abstract

The predictive advantage of combining several different predictive models is widely accepted. Particularly in time series forecasting problems, this combination is often dynamic to cope with potential non-stationary sources of variation present in the data. Despite their superior predictive performance, ensemble methods entail two main limitations: high computational costs and lack of transparency. These issues often preclude the deployment of such approaches, in favour of simpler yet more efficient and reliable ones. In this paper, we leverage the idea of model compression to address this problem in time series forecasting tasks. Model compression approaches have been mostly unexplored for forecasting. Their application in time series is challenging due to the evolving nature of the data. Further, while the literature focuses on neural networks, we apply model compression to distinct types of methods. In an extensive set of experiments, we show that compressing dynamic forecasting ensembles into an individual model leads to a comparable predictive performance and a drastic reduction in computational costs. Further, the compressed individual model with best average rank is a rule-based regression model. Thus, model compression also leads to benefits in terms of model interpretability. The experiments carried in this paper are fully reproducible.