AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling

TL;DR

AutoAttention introduces an automatic method for selecting relevant field pairs in user behavior modeling, improving CTR prediction by reducing noise and computational cost through learnable weights and pruning.

Contribution

It proposes a novel auto field pair selection mechanism for attention models that automatically identifies and removes noisy field pairs, enhancing performance and efficiency.

Findings

Effective field pair selection improves CTR prediction accuracy.

AutoAttention outperforms existing methods on public and production datasets.

Low computational cost despite including more fields.

Abstract

In Click-through rate (CTR) prediction models, a user's interest is usually represented as a fixed-length vector based on her history behaviors. Recently, several methods are proposed to learn an attentive weight for each user behavior and conduct weighted sum pooling. However, these methods only manually select several fields from the target item side as the query to interact with the behaviors, neglecting the other target item fields, as well as user and context fields. Directly including all these fields in the attention may introduce noise and deteriorate the performance. In this paper, we propose a novel model named AutoAttention, which includes all item/user/context side fields as the query, and assigns a learnable weight for each field pair between behavior fields and query fields. Pruning on these field pairs via these learnable weights lead to automatic field pair selection, so as to identify and remove noisy field pairs. Though including more fields, the computation cost of AutoAttention is still low due to using a simple attention function and field pair selection. Extensive experiments on the public dataset and Tencent's production dataset demonstrate the effectiveness of the proposed approach.