HIT Model: A Hierarchical Interaction-Enhanced Two-Tower Model for Pre-Ranking Systems

TL;DR

The HIT model enhances the traditional two-tower pre-ranking architecture by integrating hierarchical interaction components, improving relevance and business metrics without sacrificing efficiency, and has been successfully deployed at Tencent.

Contribution

It introduces a hierarchical interaction-enhanced two-tower architecture with generators and multi-head representers for better modeling of user-ad relationships.

Findings

Significantly outperforms baselines in relevance metrics.

Achieves 1.66% increase in GMV and 1.55% ROI in online tests.

Maintains similar latency to vanilla two-tower models.

Abstract

Online display advertising platforms rely on pre-ranking systems to efficiently filter and prioritize candidate ads from large corpora, balancing relevance to users with strict computational constraints. The prevailing two-tower architecture, though highly efficient due to its decoupled design and pre-caching, suffers from cross-domain interaction and coarse similarity metrics, undermining its capacity to model complex user-ad relationships. In this study, we propose the Hierarchical Interaction-Enhanced Two-Tower (HIT) model, a new architecture that augments the two-tower paradigm with two key components: $\textit{generators}$ that pre-generate holistic vectors incorporating coarse-grained user-ad interactions through a dual-generator framework with a cosine-similarity-based generation loss as the training objective, and $\textit{multi-head representers}$ that project embeddings into multiple latent subspaces to capture fine-grained, multi-faceted user interests and multi-dimensional ad attributes. This design enhances modeling effectiveness without compromising inference efficiency. Extensive experiments on public datasets and large-scale online A/B testing on Tencent's advertising platform demonstrate that HIT significantly outperforms several baselines in relevance metrics, yielding a $1.66\%$ increase in Gross Merchandise Volume and a $1.55\%$ improvement in Return on Investment, alongside similar serving latency to the vanilla two-tower models. The HIT model has been successfully deployed in Tencent's online display advertising system, serving billions of impressions daily. The code is available at https://github.com/HarveyYang123/HIT_model.