ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons

TL;DR

ProbTS is a comprehensive benchmarking platform that evaluates point and distributional forecasting models across various horizons, revealing strengths and weaknesses of current approaches and guiding future research directions.

Contribution

This paper introduces ProbTS, a unified benchmark tool for assessing diverse forecasting models across multiple horizons, addressing gaps in understanding their comparative performance.

Findings

Universal models have varying effectiveness across horizons.

Methodological biases influence forecasting performance.

Current research has limitations in addressing diverse forecasting needs.

Abstract

Delivering precise point and distributional forecasts across a spectrum of prediction horizons represents a significant and enduring challenge in the application of time-series forecasting within various industries. Prior research on developing deep learning models for time-series forecasting has often concentrated on isolated aspects, such as long-term point forecasting or short-term probabilistic estimations. This narrow focus may result in skewed methodological choices and hinder the adaptability of these models to uncharted scenarios. While there is a rising trend in developing universal forecasting models, a thorough understanding of their advantages and drawbacks, especially regarding essential forecasting needs like point and distributional forecasts across short and long horizons, is still lacking. In this paper, we present ProbTS, a benchmark tool designed as a unified platform to evaluate these fundamental forecasting needs and to conduct a rigorous comparative analysis of numerous cutting-edge studies from recent years. We dissect the distinctive data characteristics arising from disparate forecasting requirements and elucidate how these characteristics can skew methodological preferences in typical research trajectories, which often fail to fully accommodate essential forecasting needs. Building on this, we examine the latest models for universal time-series forecasting and discover that our analyses of methodological strengths and weaknesses are also applicable to these universal models. Finally, we outline the limitations inherent in current research and underscore several avenues for future exploration.