Looking at CTR Prediction Again: Is Attention All You Need?

TL;DR

This paper redefines CTR prediction using economic choice models and proposes a self-attention based neural network framework, analyzing its expressiveness and aligning existing models within this framework, supported by experiments.

Contribution

It introduces a novel framework for CTR prediction based on discrete choice models and self-attention, unifying and analyzing existing models' structures.

Findings

Most existing CTR models fit the proposed framework.

The framework's expressive power and complexity are analyzed.

Experimental results validate the framework's insights.

Abstract

Click-through rate (CTR) prediction is a critical problem in web search, recommendation systems and online advertisement displaying. Learning good feature interactions is essential to reflect user's preferences to items. Many CTR prediction models based on deep learning have been proposed, but researchers usually only pay attention to whether state-of-the-art performance is achieved, and ignore whether the entire framework is reasonable. In this work, we use the discrete choice model in economics to redefine the CTR prediction problem, and propose a general neural network framework built on self-attention mechanism. It is found that most existing CTR prediction models align with our proposed general framework. We also examine the expressive power and model complexity of our proposed framework, along with potential extensions to some existing models. And finally we demonstrate and verify our insights through some experimental results on public datasets.