Top-P Masking for Cross Language Information Retrieval

TL;DR

This paper introduces Top-P Dynamic Masking for Cross Language Information Retrieval, demonstrating it outperforms Top-K masking by promoting sparser representations, inspired by Nucleus Sampling in Large Language Models.

Contribution

The paper proposes a novel Top-P Dynamic Masking scheme for CLIR, improving upon existing Top-K masking methods with better performance.

Findings

Top-P Dynamic Masking outperforms Top-K masking in CLIR tasks.

The method promotes sparser representations leading to improved retrieval accuracy.

Demonstrates the effectiveness of Nucleus Sampling-inspired masking in IR applications.

Abstract

Top-K masking schemes have been proposed as a method to promote sparse representations in Information Retrieval (IR) tasks, as a simple alternative to Floating Point Operations per Second (FLOPS) regularization. Algorithms such as Bilingual Lexical and Document Expansion Model (BLADE), adopt this approach as a post-processing stage. We propose using Top-P Dynamic Masking similar to Nucleus Sampling in Large Language Models, and demonstrate better performance than Top-K masking. Specifically, we evaluate our methods in the domain of Cross Language Information Retrieval (CLIR)