AdaSparse: Learning Adaptively Sparse Structures for Multi-Domain Click-Through Rate Prediction

TL;DR

AdaSparse is a novel framework that learns adaptively sparse neural structures for multi-domain CTR prediction, enhancing generalization and reducing computational costs through domain-aware neuron weighting and regularization.

Contribution

It introduces a simple, effective method for learning domain-specific sparse neural structures, improving multi-domain CTR prediction performance and efficiency.

Findings

Outperforms previous multi-domain CTR models significantly.

Achieves better generalization with lower computational cost.

Effectively prunes redundant neurons across domains.

Abstract

Click-through rate (CTR) prediction is a fundamental technique in recommendation and advertising systems. Recent studies have proved that learning a unified model to serve multiple domains is effective to improve the overall performance. However, it is still challenging to improve generalization across domains under limited training data, and hard to deploy current solutions due to their computational complexity. In this paper, we propose a simple yet effective framework AdaSparse for multi-domain CTR prediction, which learns adaptively sparse structure for each domain, achieving better generalization across domains with lower computational cost. In AdaSparse, we introduce domain-aware neuron-level weighting factors to measure the importance of neurons, with that for each domain our model can prune redundant neurons to improve generalization. We further add flexible sparsity regularizations to control the sparsity ratio of learned structures. Offline and online experiments show that AdaSparse outperforms previous multi-domain CTR models significantly.