Deep Continuous-Time State-Space Models for Marked Event Sequences

TL;DR

This paper introduces S2P2, a deep continuous-time state-space model for marked event sequences that improves predictive performance and scalability over existing models by combining stochastic differential equations with neural networks.

Contribution

The paper presents a novel deep state-space model for MTPPs that captures continuous-time dynamics without restrictive assumptions, enabling efficient training and superior predictive accuracy.

Findings

Achieves state-of-the-art likelihoods on eight datasets

Improves predictive likelihood by 33% on average

Supports efficient linear-time training and inference

Abstract

Marked temporal point processes (MTPPs) model sequences of events occurring at irregular time intervals, with wide-ranging applications in fields such as healthcare, finance and social networks. We propose the state-space point process (S2P2) model, a novel and performant model that leverages techniques derived for modern deep state-space models (SSMs) to overcome limitations of existing MTPP models, while simultaneously imbuing strong inductive biases for continuous-time event sequences that other discrete sequence models (i.e., RNNs, transformers) do not capture. Inspired by the classical linear Hawkes processes, we propose an architecture that interleaves stochastic jump differential equations with nonlinearities to create a highly expressive intensity-based MTPP model, without the need for restrictive parametric assumptions for the intensity. Our approach enables efficient training and inference with a parallel scan, bringing linear complexity and sublinear scaling while retaining expressivity to MTPPs. Empirically, S2P2 achieves state-of-the-art predictive likelihoods across eight real-world datasets, delivering an average improvement of 33% over the best existing approaches.