STEC: See-Through Transformer-based Encoder for CTR Prediction

TL;DR

STEC is a novel transformer-based encoder that integrates multiple interaction learning strategies with residual connections, significantly improving CTR prediction accuracy on real-world datasets.

Contribution

The paper introduces STEC, a unified model combining multiple interaction strategies with residual connections for enhanced CTR prediction performance.

Findings

STEC outperforms existing state-of-the-art models on four datasets.

Residual connections from different interaction orders boost prediction accuracy.

The model demonstrates greater expressive capabilities in modeling user-item interactions.

Abstract

Click-Through Rate (CTR) prediction holds a pivotal place in online advertising and recommender systems since CTR prediction performance directly influences the overall satisfaction of the users and the revenue generated by companies. Even so, CTR prediction is still an active area of research since it involves accurately modelling the preferences of users based on sparse and high-dimensional features where the higher-order interactions of multiple features can lead to different outcomes. Most CTR prediction models have relied on a single fusion and interaction learning strategy. The few CTR prediction models that have utilized multiple interaction modelling strategies have treated each interaction to be self-contained. In this paper, we propose a novel model named STEC that reaps the benefits of multiple interaction learning approaches in a single unified architecture. Additionally, our model introduces residual connections from different orders of interactions which boosts the performance by allowing lower level interactions to directly affect the predictions. Through extensive experiments on four real-world datasets, we demonstrate that STEC outperforms existing state-of-the-art approaches for CTR prediction thanks to its greater expressive capabilities.