Pretext Training Algorithms for Event Sequence Data

TL;DR

This paper introduces a self-supervised pretext training framework for event sequence data, utilizing a novel alignment verification task to learn versatile representations applicable to various downstream tasks.

Contribution

It presents a new pretext training method specifically designed for event sequences, combining masked reconstruction and contrastive learning to improve downstream task performance.

Findings

Effective across multiple event sequence tasks

Achieves strong results on public benchmarks

Uncovers foundational representations for event data

Abstract

Pretext training followed by task-specific fine-tuning has been a successful approach in vision and language domains. This paper proposes a self-supervised pretext training framework tailored to event sequence data. We introduce a novel alignment verification task that is specialized to event sequences, building on good practices in masked reconstruction and contrastive learning. Our pretext tasks unlock foundational representations that are generalizable across different down-stream tasks, including next-event prediction for temporal point process models, event sequence classification, and missing event interpolation. Experiments on popular public benchmarks demonstrate the potential of the proposed method across different tasks and data domains.