Universal Transformer Hawkes Process with Adaptive Recursive Iteration

TL;DR

This paper introduces the Universal Transformer Hawkes Process (UTHP), combining recursive mechanisms, self-attention, and CNNs to enhance modeling of asynchronous event sequences, achieving improved performance over existing models.

Contribution

The paper proposes a novel universal transformer Hawkes process model with recursive and self-attention mechanisms, plus CNN integration, to better capture local and global features in event data.

Findings

UTHP outperforms previous state-of-the-art models on multiple datasets.

Recursive mechanisms improve model performance.

Incorporating CNN enhances local perception in the model.

Abstract

Asynchronous events sequences are widely distributed in the natural world and human activities, such as earthquakes records, users activities in social media and so on. How to distill the information from these seemingly disorganized data is a persistent topic that researchers focus on. The one of the most useful model is the point process model, and on the basis, the researchers obtain many noticeable results. Moreover, in recent years, point process models on the foundation of neural networks, especially recurrent neural networks (RNN) are proposed and compare with the traditional models, their performance are greatly improved. Enlighten by transformer model, which can learning sequence data efficiently without recurrent and convolutional structure, transformer Hawkes process is come out, and achieves state-of-the-art performance. However, there is some research proving that the re-introduction of recursive calculations in transformer can further improve transformers performance. Thus, we come out with a new kind of transformer Hawkes process model, universal transformer Hawkes process (UTHP), which contains both recursive mechanism and self-attention mechanism, and to improve the local perception ability of the model, we also introduce convolutional neural network (CNN) in the position-wise-feed-forward part. We conduct experiments on several datasets to validate the effectiveness of UTHP and explore the changes after the introduction of the recursive mechanism. These experiments on multiple datasets demonstrate that the performance of our proposed new model has a certain improvement compared with the previous state-of-the-art models.