HMDN: Hierarchical Multi-Distribution Network for Click-Through Rate Prediction

TL;DR

HMDN introduces a hierarchical multi-distribution modeling framework that captures complex relationships in diverse recommendation scenarios, enhancing existing multi-distribution models' flexibility and effectiveness.

Contribution

The paper proposes a novel Hierarchical Multi-Distribution Network (HMDN) that models hierarchical relationships among distributions and integrates with existing methods like MoE and DW.

Findings

HMDN improves prediction accuracy on public datasets.

HMDN demonstrates effectiveness on industrial datasets.

The framework is flexible and compatible with existing models.

Abstract

As the recommendation service needs to address increasingly diverse distributions, such as multi-population, multi-scenario, multitarget, and multi-interest, more and more recent works have focused on multi-distribution modeling and achieved great progress. However, most of them only consider modeling in a single multi-distribution manner, ignoring that mixed multi-distributions often coexist and form hierarchical relationships. To address these challenges, we propose a flexible modeling paradigm, named Hierarchical Multi-Distribution Network (HMDN), which efficiently models these hierarchical relationships and can seamlessly integrate with existing multi-distribution methods, such as Mixture of-Experts (MoE) and Dynamic-Weight (DW) models. Specifically, we first design a hierarchical multi-distribution representation refinement module, employing a multi-level residual quantization to obtain fine-grained hierarchical representation. Then, the refined hierarchical representation is integrated into the existing single multi-distribution models, seamlessly expanding them into mixed multi-distribution models. Experimental results on both public and industrial datasets validate the effectiveness and flexibility of HMDN.