TBIN: Modeling Long Textual Behavior Data for CTR Prediction

TL;DR

This paper introduces TBIN, a novel model that effectively captures long and diverse user interests from textual behavior data for improved CTR prediction, overcoming computational and representational limitations of previous methods.

Contribution

TBIN combines locality-sensitive hashing and shifted chunk-based self-attention to model long user behavior data efficiently and dynamically activate diverse interests for better CTR prediction.

Findings

TBIN outperforms existing models in offline experiments.

TBIN achieves significant online performance improvements.

The model effectively captures long-term user interests.

Abstract

Click-through rate (CTR) prediction plays a pivotal role in the success of recommendations. Inspired by the recent thriving of language models (LMs), a surge of works improve prediction by organizing user behavior data in a \textbf{textual} format and using LMs to understand user interest at a semantic level. While promising, these works have to truncate the textual data to reduce the quadratic computational overhead of self-attention in LMs. However, it has been studied that long user behavior data can significantly benefit CTR prediction. In addition, these works typically condense user diverse interests into a single feature vector, which hinders the expressive capability of the model. In this paper, we propose a \textbf{T}extual \textbf{B}ehavior-based \textbf{I}nterest Chunking \textbf{N}etwork (TBIN), which tackles the above limitations by combining an efficient locality-sensitive hashing algorithm and a shifted chunk-based self-attention. The resulting user diverse interests are dynamically activated, producing user interest representation towards the target item. Finally, the results of both offline and online experiments on real-world food recommendation platform demonstrate the effectiveness of TBIN.