AdnFM: An Attentive DenseNet based Factorization Machine for CTR Prediction

TL;DR

This paper introduces AdnFM, a novel model combining DenseNet-inspired neural networks with attention mechanisms to enhance high-order feature interaction modeling for CTR prediction, achieving improved performance on real-world datasets.

Contribution

It proposes AdnFM, a new model that leverages all hidden layers and attention to better capture high-order feature interactions efficiently.

Findings

AdnFM outperforms existing CTR prediction models on real-world datasets.

Using all hidden layers improves the extraction of deep feature interactions.

Attention mechanism effectively selects dominant features for better prediction.

Abstract

In this paper, we consider the Click-Through-Rate (CTR) prediction problem. Factorization Machines and their variants consider pair-wise feature interactions, but normally we won't do high-order feature interactions using FM due to high time complexity. Given the success of deep neural networks (DNNs) in many fields, researchers have proposed several DNN-based models to learn high-order feature interactions. Multi-layer perceptrons (MLP) have been widely employed to learn reliable mappings from feature embeddings to final logits. In this paper, we aim to explore more about these high-order features interactions. However, high-order feature interaction deserves more attention and further development. Inspired by the great achievements of Densely Connected Convolutional Networks (DenseNet) in computer vision, we propose a novel model called Attentive DenseNet based Factorization Machines (AdnFM). AdnFM can extract more comprehensive deep features by using all the hidden layers from a feed-forward neural network as implicit high-order features, then selects dominant features via an attention mechanism. Also, high-order interactions in the implicit way using DNNs are more cost-efficient than in the explicit way, for example in FM. Extensive experiments on two real-world datasets show that the proposed model can effectively improve the performance of CTR prediction.