A Collaborative Ensemble Framework for CTR Prediction

TL;DR

This paper introduces CETNet, a collaborative ensemble framework for CTR prediction that leverages diverse models with confidence-based fusion, outperforming individual models and state-of-the-art baselines on multiple datasets.

Contribution

The paper proposes a novel collaborative ensemble training framework that enhances CTR prediction by combining diverse models with confidence-based dynamic weighting.

Findings

CETNet outperforms individual models and baselines on multiple datasets.

The framework achieves comparable or better results with smaller embedding sizes.

Experimental results validate the effectiveness and scalability of CETNet.

Abstract

Recent advances in foundation models have established scaling laws that enable the development of larger models to achieve enhanced performance, motivating extensive research into large-scale recommendation models. However, simply increasing the model size in recommendation systems, even with large amounts of data, does not always result in the expected performance improvements. In this paper, we propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models, each with its own embedding table, to capture unique feature interaction patterns. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning, where models iteratively refine their predictions. To dynamically balance contributions from each model, we introduce a confidence-based fusion mechanism using general softmax, where model confidence is computed via negation entropy. This design ensures that more confident models have a greater influence on the final prediction while benefiting from the complementary strengths of other models. We validate our framework on three public datasets (AmazonElectronics, TaobaoAds, and KuaiVideo) as well as a large-scale industrial dataset from Meta, demonstrating its superior performance over individual models and state-of-the-art baselines. Additionally, we conduct further experiments on the Criteo and Avazu datasets to compare our method with the multi-embedding paradigm. Our results show that our framework achieves comparable or better performance with smaller embedding sizes, offering a scalable and efficient solution for CTR prediction tasks.