HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?

TL;DR

This paper introduces HoTPP, a benchmark for evaluating long-horizon event forecasting using Marked Temporal Point Processes, revealing that current methods often underperform simple baselines and suffer from mode collapse.

Contribution

We present the first benchmark specifically for long-horizon event forecasting with MTPP, propose a new evaluation metric, and provide comprehensive empirical analysis and insights.

Findings

Modern MTPP methods often underperform simple baselines.

Most methods exhibit mode collapse in sequence diversity.

Autoregression and intensity-based losses impact prediction quality.

Abstract

Forecasting multiple future events within a given time horizon is essential for applications in finance, retail, social networks, and healthcare. Marked Temporal Point Processes (MTPP) provide a principled framework to model both the timing and labels of events. However, most existing research focuses on predicting only the next event, leaving long-horizon forecasting largely underexplored. To address this gap, we introduce HoTPP, the first benchmark specifically designed to rigorously evaluate long-horizon predictions. We identify shortcomings in widely used evaluation metrics, propose a theoretically grounded T-mAP metric, present strong statistical baselines, and offer efficient implementations of popular models. Our empirical results demonstrate that modern MTPP approaches often underperform simple statistical baselines. Furthermore, we analyze the diversity of predicted sequences and find that most methods exhibit mode collapse. Finally, we analyze the impact of autoregression and intensity-based losses on prediction quality, and outline promising directions for future research. The HoTPP source code, hyperparameters, and full evaluation results are available at GitHub.