MMBAttn: Max-Mean and Bit-wise Attention for CTR Prediction

TL;DR

This paper introduces MMBAttn, an attention mechanism combining max-mean pooling and bit-wise attention to improve feature importance estimation in CTR prediction, achieving state-of-the-art results.

Contribution

It presents a novel attention architecture that leverages bit-wise interactions along with max and mean pooling for better feature importance estimation in CTR models.

Findings

Significantly improves CTR prediction accuracy

Achieves state-of-the-art results on public datasets

Enhances feature importance estimation with bit-level attention

Abstract

With the increasing complexity and scale of click-through rate (CTR) prediction tasks in online advertising and recommendation systems, accurately estimating the importance of features has become a critical aspect of developing effective models. In this paper, we propose an attention-based approach that leverages max and mean pooling operations, along with a bit-wise attention mechanism, to enhance feature importance estimation in CTR prediction. Traditionally, pooling operations such as max and mean pooling have been widely used to extract relevant information from features. However, these operations can lead to information loss and hinder the accurate determination of feature importance. To address this challenge, we propose a novel attention architecture that utilizes a bit-based attention structure that emphasizes the relationships between all bits in features, together with maximum and mean pooling. By considering the fine-grained interactions at the bit level, our method aims to capture intricate patterns and dependencies that might be overlooked by traditional pooling operations. To examine the effectiveness of the proposed method, experiments have been conducted on three public datasets. The experiments demonstrated that the proposed method significantly improves the performance of the base models to achieve state-of-the-art results.