DEPHN: Different Expression Parallel Heterogeneous Network using virtual gradient optimization for Multi-task Learning

TL;DR

DEPHN introduces a novel multi-task learning network that models complex task relationships using diverse feature interactions and adaptive gradient optimization, enhancing recommendation system performance.

Contribution

The paper proposes DEPHN, a new multi-task learning model with heterogeneous experts and virtual gradient optimization to better capture complex task correlations.

Findings

DEPHN outperforms baseline models on artificial and real-world datasets.

The model effectively captures complex task relationships.

Adaptive gradient optimization improves learning efficiency.

Abstract

Recommendation system algorithm based on multi-task learning (MTL) is the major method for Internet operators to understand users and predict their behaviors in the multi-behavior scenario of platform. Task correlation is an important consideration of MTL goals, traditional models use shared-bottom models and gating experts to realize shared representation learning and information differentiation. However, The relationship between real-world tasks is often more complex than existing methods do not handle properly sharing information. In this paper, we propose an Different Expression Parallel Heterogeneous Network (DEPHN) to model multiple tasks simultaneously. DEPHN constructs the experts at the bottom of the model by using different feature interaction methods to improve the generalization ability of the shared information flow. In view of the model's differentiating ability for different task information flows, DEPHN uses feature explicit mapping and virtual gradient coefficient for expert gating during the training process, and adaptively adjusts the learning intensity of the gated unit by considering the difference of gating values and task correlation. Extensive experiments on artificial and real-world datasets demonstrate that our proposed method can capture task correlation in complex situations and achieve better performance than baseline models\footnote{Accepted in IJCNN2023}.