TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond

TL;DR

TraXion introduces a novel pre-training framework tailored for multi-entity spatiotemporal event streams, outperforming task-specific models across diverse domains like mobility, security, and healthcare.

Contribution

The paper defines three axioms for MESES and designs TraXion to satisfy them, enabling a unified pre-training approach for various event-stream domains.

Findings

TraXion outperforms task-specific baselines on mobility datasets.

It matches or exceeds prior work in enterprise authentication and healthcare.

A single checkpoint works across multiple domains and tasks.

Abstract

Human mobility differs from text and from generic time series in three structural ways: visits are tuple-valued events whose meaning depends on the joint distribution over location, time, and activity; users carry persistent signatures across trajectories; and visits are not independent across users, since co-location at shared places is a primary signal. Existing pre-training recipes for mobility import objectives from language modeling, treating trajectories as sentences and visits as tokens, an analogy that fails against each of the three properties above. These properties define a broader class, multi-entity spatiotemporal event streams (MESES), spanning enterprise authentication logs, electronic health records, and other event-stream domains where entities share infrastructure, schedules, or contexts. We make the properties precise as three axioms that any pre-training framework for MESES should satisfy, and introduce TraXion, whose objectives and architecture are jointly designed to meet them. A single TraXion checkpoint per dataset beats task-specific baselines on every task across six public mobility datasets covering anomaly detection, next-POI recommendation, next-visit prediction, and social-link prediction. The same recipe, applied unchanged to enterprise authentication logs and ICU mortality prediction, matches or exceeds prior work on both, showing that event streams from domains as different as mobility, security, and healthcare can be modeled under a single framework.