Self-Attentive Hawkes Processes

TL;DR

The paper introduces a self-attentive Hawkes process (SAHP) that enhances modeling of asynchronous event sequences by capturing complex dependencies and longer historical information with improved interpretability over traditional methods.

Contribution

It proposes a novel self-attention based extension to Hawkes processes, improving dependency modeling and interpretability in event sequence analysis.

Findings

SAHP outperforms traditional Hawkes models in capturing complex dependencies.

SAHP demonstrates better long-term dependency modeling than recurrent networks.

Experimental results on real-world datasets validate the effectiveness of SAHP.

Abstract

Asynchronous events on the continuous time domain, e.g., social media actions and stock transactions, occur frequently in the world. The ability to recognize occurrence patterns of event sequences is crucial to predict which typeof events will happen next and when. A de facto standard mathematical framework to do this is the Hawkes process. In order to enhance expressivity of multivariate Hawkes processes, conventional statistical methods and deep recurrent networks have been employed to modify its intensity function. The former is highly interpretable and requires small size of training data but relies on correct model design while the latter has less dependency on prior knowledge and is more powerful in capturing complicated patterns. We leverage pros and cons of these models and propose a self-attentive Hawkes process(SAHP). The proposed method adapts self-attention to fit the intensity function of Hawkes processes. This design has two benefits:(1) compared with conventional statistical methods, the SAHP is more powerful to identify complicated dependency relationships between temporal events; (2)compared with deep recurrent networks, the self-attention mechanism is able to capture longer historical information, and is more interpretable because the learnt attention weight tensor shows contributions of each historical event. Experiments on four real-world datasets demonstrate the effectiveness of the proposed method.