Deep Multi-Representation Model for Click-Through Rate Prediction

TL;DR

The paper introduces DeepMR, a novel CTR prediction model that combines DNNs and multi-head self-attention with ReZero connections, achieving superior performance on real-world datasets.

Contribution

DeepMR is the first model to jointly train DNNs and multi-head self-attention with ReZero, enhancing feature representation for CTR prediction.

Findings

DeepMR outperforms state-of-the-art models on three datasets.

ReZero connections improve training stability and representation quality.

Joint training of DNN and self-attention components enhances prediction accuracy.

Abstract

Click-Through Rate prediction (CTR) is a crucial task in recommender systems, and it gained considerable attention in the past few years. The primary purpose of recent research emphasizes obtaining meaningful and powerful representations through mining low and high feature interactions using various components such as Deep Neural Networks (DNN), CrossNets, or transformer blocks. In this work, we propose the Deep Multi-Representation model (DeepMR) that jointly trains a mixture of two powerful feature representation learning components, namely DNNs and multi-head self-attentions. Furthermore, DeepMR integrates the novel residual with zero initialization (ReZero) connections to the DNN and the multi-head self-attention components for learning superior input representations. Experiments on three real-world datasets show that the proposed model significantly outperforms all state-of-the-art models in the task of click-through rate prediction.