Tri-Transformer Hawkes Process: Three Heads are better than one

TL;DR

This paper introduces the Tri-Transformer Hawkes Process, a novel model that enhances event sequence modeling by integrating three transformer heads to better utilize event time and type information, outperforming previous methods.

Contribution

The paper proposes a tri-transformer architecture that effectively incorporates event and time information, reducing learning bias and improving performance over existing transformer Hawkes processes.

Findings

Tri-THP outperforms existing models on real-world data.

Incorporating multiple transformer heads improves learning accuracy.

The model effectively utilizes event time and type information.

Abstract

Abstract. Most of the real world data we encounter are asynchronous event sequence, so the last decades have been characterized by the implementation of various point process into the field of social networks,electronic medical records and financial transactions. At the beginning, Hawkes process and its variants which can simulate simultaneously the self-triggering and mutual triggering patterns between different events in complex sequences in a clear and quantitative way are more popular.Later on, with the advances of neural network, neural Hawkes process has been proposed one after another, and gradually become a research hotspot. The proposal of the transformer Hawkes process (THP) has gained a huge performance improvement, so a new upsurge of the neural Hawkes process based on transformer is set off. However, THP does not make full use of the information of occurrence time and type of event in the asynchronous event sequence. It simply adds the encoding of event type conversion and the location encoding of time conversion to the source encoding. At the same time, the learner built from a single transformer will result in an inescapable learning bias. In order to mitigate these problems, we propose a tri-transformer Hawkes process (Tri-THP) model, in which the event and time information are added to the dot-product attention as auxiliary information to form a new multihead attention. The effectiveness of the Tri-THP is proved by a series of well-designed experiments on both real world and synthetic data.