DCAP: Deep Cross Attentional Product Network for User Response Prediction

TL;DR

DCAP is a novel deep learning architecture that explicitly models high-order feature interactions with attention mechanisms, improving user response prediction in sparse, high-dimensional data environments.

Contribution

Introduces DCAP, a deep cross attentional product network that differentiates feature importance across layers, enhancing modeling of high-order interactions for user response prediction.

Findings

DCAP outperforms state-of-the-art models on three real-world datasets.

The model effectively captures the importance of different cross features.

Parallel training makes DCAP scalable and practical.

Abstract

User response prediction, which aims to predict the probability that a user will provide a predefined positive response in a given context such as clicking on an ad or purchasing an item, is crucial to many industrial applications such as online advertising, recommender systems, and search ranking. However, due to the high dimensionality and super sparsity of the data collected in these tasks, handcrafting cross features is inevitably time expensive. Prior studies in predicting user response leveraged the feature interactions by enhancing feature vectors with products of features to model second-order or high-order cross features, either explicitly or implicitly. Nevertheless, these existing methods can be hindered by not learning sufficient cross features due to model architecture limitations or modeling all high-order feature interactions with equal weights. This work aims to fill this gap by proposing a novel architecture Deep Cross Attentional Product Network (DCAP), which keeps cross network's benefits in modeling high-order feature interactions explicitly at the vector-wise level. Beyond that, it can differentiate the importance of different cross features in each network layer inspired by the multi-head attention mechanism and Product Neural Network (PNN), allowing practitioners to perform a more in-depth analysis of user behaviors. Additionally, our proposed model can be easily implemented and train in parallel. We conduct comprehensive experiments on three real-world datasets. The results have robustly demonstrated that our proposed model DCAP achieves superior prediction performance compared with the state-of-the-art models. Public codes are available at https://github.com/zachstarkk/DCAP.