Towards Measuring Fairness in Grid Layout in Recommender Systems

TL;DR

This paper investigates how existing fairness metrics for linear recommender rankings can be adapted and applied to grid-based layouts, considering device-dependent display variations and their impact on fairness measurement.

Contribution

It extends fair ranking metrics to grid layouts, analyzes their behavior across different designs, and offers insights for applying fairness metrics in practical, device-adaptive recommender systems.

Findings

Fairness scores vary with layout design and device geometry.

Linear fairness metrics may not directly translate to grid layouts.

Layout-specific user attention models are necessary for accurate fairness measurement.

Abstract

There has been significant research in the last five years on ensuring the providers of items in a recommender system are treated fairly, particularly in terms of the exposure the system provides to their work through its results. However, the metrics developed to date have all been designed and tested for linear ranked lists. It is unknown whether and how existing fair ranking metrics for linear layouts can be applied to grid-based displays. Moreover, depending on the device (phone, tab, or laptop) users use to interact with systems, column size is adjusted using column reduction approaches in a grid-view. The visibility or exposure of recommended items in grid layouts varies based on column sizes and column reduction approaches as well. In this paper, we extend existing fair ranking concepts and metrics to study provider-side group fairness in grid layouts, present an analysis of the behavior of these grid adaptations of fair ranking metrics, and study how their behavior changes across different grid ranking layout designs and geometries. We examine how fairness scores change with different ranking layouts to yield insights into (1) the consistency of fair ranking measurements across layouts; (2) whether rankings optimized for fairness in a linear ranking remain fair when the results are displayed in a grid; and (3) the impact of column reduction approaches to support different device geometries on fairness measurement. This work highlights the need to use layout-specific user attention models when measuring fairness of rankings, and provide practitioners with a first set of insights on what to expect when translating existing fair ranking metrics to the grid layouts in wide use today.