# T-Retrievability: A Topic-Focused Approach to Measure Fair Document Exposure in Information Retrieval

## TL;DR

This paper introduces T-Retrievability, a topic-focused metric for assessing fairness in document exposure by analyzing localized retrievability scores across topical groups, providing deeper insights into model bias.

## Contribution

The work proposes a novel topic-focused retrievability measure that improves the assessment of exposure bias in IR models by considering topical relevance groups.

## Key findings

- Uncovers nuanced exposure characteristics of neural ranking models
- Demonstrates the effectiveness of localized retrievability in fairness assessment
- Provides a more reliable measure for document accessibility in IR systems

## Abstract

Retrievability of a document is a collection-based statistic that measures its expected (reciprocal) rank of being retrieved within a specific rank cut-off. A collection with uniformly distributed retrievability scores across documents is an indicator of fair document exposure. While retrievability scores have been used to quantify the fairness of exposure for a collection, in our work, we use the distribution of retrievability scores to measure the exposure bias of retrieval models. We hypothesise that an uneven distribution of retrievability scores across the entire collection may not accurately reflect exposure bias but rather indicate variations in topical relevance. As a solution, we propose a topic-focused localised retrievability measure, which we call \textit{T-Retrievability} (topic-retrievability), which first computes retrievability scores over multiple groups of topically-related documents, and then aggregates these localised values to obtain the collection-level statistics. Our analysis using this proposed T-Retrievability measure uncovers new insights into the exposure characteristics of various neural ranking models. The findings suggest that this localised measure provides a more nuanced understanding of exposure fairness, offering a more reliable approach for assessing document accessibility in IR systems.