Denoising Attention for Query-aware User Modeling in Personalized Search

TL;DR

This paper introduces Denoising Attention, a novel variant designed to improve query-aware user modeling in personalized search by addressing noise filtering and normalization issues in standard Attention mechanisms.

Contribution

The paper proposes Denoising Attention, a new attention mechanism with robust normalization and filtering to enhance personalization in search systems.

Findings

Denoising Attention outperforms standard and other Attention variants in experiments.

The approach effectively filters noisy user information.

Improves relevance of personalized search results.

Abstract

The personalization of search results has gained increasing attention in the past few years, thanks to the development of Neural Networks-based approaches for Information Retrieval and the importance of personalization in many search scenarios. Recent works have proposed to build user models at query time by leveraging the Attention mechanism, which allows weighing the contribution of the user-related information w.r.t. the current query. This approach allows taking into account the diversity of the user's interests by giving more importance to those related to the current search performed by the user. In this paper, we first discuss some shortcomings of the standard Attention formulation when employed for personalization. In particular, we focus on issues related to its normalization mechanism and its inability to entirely filter out noisy user-related information. Then, we introduce the Denoising Attention mechanism: an Attention variant that directly tackles the above shortcomings by adopting a robust normalization scheme and introducing a filtering mechanism. The reported experimental evaluation shows the benefits of the proposed approach over other Attention-based variants.