TRACE: Transformer-based user Representations from Attributed Clickstream Event sequences

TL;DR

TRACE is a transformer-based method that creates detailed user embeddings from multi-session clickstream data, improving real-time personalized recommendations in travel e-commerce by capturing long-term user behaviors.

Contribution

It introduces a novel transformer approach that models site-wide, multi-session user journeys, unlike prior single-session focused methods, for enhanced user preference understanding.

Findings

TRACE outperforms vanilla transformer and LLM architectures in experiments.

Learned embeddings reveal meaningful user behavior clusters.

The method effectively captures long-term user engagement patterns.

Abstract

For users navigating travel e-commerce websites, the process of researching products and making a purchase often results in intricate browsing patterns that span numerous sessions over an extended period of time. The resulting clickstream data chronicle these user journeys and present valuable opportunities to derive insights that can significantly enhance personalized recommendations. We introduce TRACE, a novel transformer-based approach tailored to generate rich user embeddings from live multi-session clickstreams for real-time recommendation applications. Prior works largely focus on single-session product sequences, whereas TRACE leverages site-wide page view sequences spanning multiple user sessions to model long-term engagement. Employing a multi-task learning framework, TRACE captures comprehensive user preferences and intents distilled into low-dimensional representations. We demonstrate TRACE's superior performance over vanilla transformer and LLM-style architectures through extensive experiments on a large-scale travel e-commerce dataset of real user journeys, where the challenges of long page-histories and sparse targets are particularly prevalent. Visualizations of the learned embeddings reveal meaningful clusters corresponding to latent user states and behaviors, highlighting TRACE's potential to enhance recommendation systems by capturing nuanced user interactions and preferences