MemoNet: Memorizing All Cross Features' Representations Efficiently via Multi-Hash Codebook Network for CTR Prediction

TL;DR

MemoNet introduces an efficient memory mechanism using multi-hash codebooks to memorize cross feature representations, significantly improving CTR prediction performance and demonstrating scalable benefits similar to large language models.

Contribution

The paper proposes HCNet, a novel multi-hash codebook memory mechanism, integrated into MemoNet for better cross feature representation learning in CTR models.

Findings

MemoNet outperforms state-of-the-art methods on three datasets.

Enlarging the codebook size leads to continuous performance improvements.

MemoNet exhibits a scaling law similar to large language models.

Abstract

New findings in natural language processing (NLP) demonstrate that the strong memorization capability contributes a lot to the success of Large Language Models (LLM). This inspires us to explicitly bring an independent memory mechanism into CTR ranking model to learn and memorize cross features' representations. In this paper, we propose multi-Hash Codebook NETwork (HCNet) as the memory mechanism for efficiently learning and memorizing representations of cross features in CTR tasks. HCNet uses a multi-hash codebook as the main memory place and the whole memory procedure consists of three phases: multi-hash addressing, memory restoring, and feature shrinking. We also propose a new CTR model named MemoNet which combines HCNet with a DNN backbone. Extensive experimental results on three public datasets and online test show that MemoNet reaches superior performance over state-of-the-art approaches. Besides, MemoNet shows scaling law of large language model in NLP, which means we can enlarge the size of the codebook in HCNet to sustainably obtain performance gains. Our work demonstrates the importance and feasibility of learning and memorizing representations of cross features, which sheds light on a new promising research direction.