PHN: Parallel heterogeneous network with soft gating for CTR prediction

TL;DR

This paper introduces PHN, a parallel heterogeneous network with soft gating and residual links to improve CTR prediction, addressing training efficiency and gradient issues in complex parallel models.

Contribution

The paper proposes a novel PHN model with soft selection gating and residual connections, enhancing training stability and effectiveness for CTR prediction.

Findings

PHN outperforms baseline models in CTR prediction accuracy.

The residual links mitigate weak gradient problems during training.

Visualization shows improved training dynamics with PHN.

Abstract

The Click-though Rate (CTR) prediction task is a basic task in recommendation system. Most of the previous researches of CTR models built based on Wide \& deep structure and gradually evolved into parallel structures with different modules. However, the simple accumulation of parallel structures can lead to higher structural complexity and longer training time. Based on the Sigmoid activation function of output layer, the linear addition activation value of parallel structures in the training process is easy to make the samples fall into the weak gradient interval, resulting in the phenomenon of weak gradient, and reducing the effectiveness of training. To this end, this paper proposes a Parallel Heterogeneous Network (PHN) model, which constructs a network with parallel structure through three different interaction analysis methods, and uses Soft Selection Gating (SSG) to feature heterogeneous data with different structure. Finally, residual link with trainable parameters are used in the network to mitigate the influence of weak gradient phenomenon. Furthermore, we demonstrate the effectiveness of PHN in a large number of comparative experiments, and visualize the performance of the model in training process and structure.