Behavior Structformer: Learning Players Representations with Structured Tokenization

TL;DR

The paper presents Behavior Structformer, a Transformer-based method that uses structured tokenization of user behavior data to improve modeling efficiency and performance over traditional methods.

Contribution

It introduces structured tokenization within a Transformer architecture for user behavior modeling, demonstrating improved effectiveness and training efficiency.

Findings

Outperforms traditional tabular and semi-structured baselines

Enhances training efficiency and effectiveness

Significantly improves behavior modeling accuracy

Abstract

In this paper, we introduce the Behavior Structformer, a method for modeling user behavior using structured tokenization within a Transformer-based architecture. By converting tracking events into dense tokens, this approach enhances model training efficiency and effectiveness. We demonstrate its superior performance through ablation studies and benchmarking against traditional tabular and semi-structured baselines. The results indicate that structured tokenization with sequential processing significantly improves behavior modeling.