DexDeepFM: Ensemble Diversity Enhanced Extreme Deep Factorization Machine Model

TL;DR

DexDeepFM enhances the extreme deep factorization machine by incorporating ensemble diversity measures and attention mechanisms, leading to improved prediction accuracy and reduced overfitting in user response prediction tasks.

Contribution

The paper introduces DexDeepFM, which integrates ensemble diversity and attention mechanisms into xDeepFM to improve feature interaction modeling and prediction performance.

Findings

Outperforms baseline models on three real-world datasets.

Ensemble diversity measures improve model robustness.

Attention mechanism effectively weights feature interaction orders.

Abstract

Predicting user positive response (e.g., purchases and clicks) probability is a critical task in Web applications. To identify predictive features from raw data, the state-of-the-art extreme deep factorization machine model (xDeepFM) introduces a new interaction network to leverage feature interactions at the vector-wise level explicitly. However, since each hidden layer in the interaction network is a collection of feature maps, it can be viewed essentially as an ensemble of different feature maps. In this case, only using a single objective to minimize the prediction loss may lead to overfitting and generate correlated errors. In this paper, an ensemble diversity enhanced extreme deep factorization machine model (DexDeepFM) is proposed, which designs the ensemble diversity measure in each hidden layer and considers both ensemble diversity and prediction accuracy in the objective function. In addition, the attention mechanism is introduced to discriminate the importance of ensemble diversity measures with different feature interaction orders. Extensive experiments on three public real-world datasets are conducted to show the effectiveness of the proposed model.