Investigating Bias and Fairness in Appearance-based Gaze Estimation

TL;DR

This study evaluates fairness in appearance-based gaze estimation, revealing significant biases across demographics and limited mitigation effectiveness, and provides resources for future research.

Contribution

It is the first comprehensive benchmark analyzing demographic bias in gaze estimation and assesses bias mitigation strategies' effectiveness.

Findings

Significant performance disparities across ethnicity and gender.

Existing bias mitigation strategies have limited impact.

Public release of annotations, code, and models to support future research.

Abstract

While appearance-based gaze estimation has achieved significant improvements in accuracy and domain adaptation, the fairness of these systems across different demographic groups remains largely unexplored. To date, there is no comprehensive benchmark quantifying algorithmic bias in gaze estimation. This paper presents the first extensive evaluation of fairness in appearance-based gaze estimation, focusing on ethnicity and gender attributes. We establish a fairness baseline by analyzing state-of-the-art models using standard fairness metrics, revealing significant performance disparities. Furthermore, we evaluate the effectiveness of existing bias mitigation strategies when applied to the gaze domain and show that their fairness contributions are limited. We summarize key insights and open issues. Overall, our work calls for research into developing robust, equitable gaze estimators. To support future research and reproducibility, we publicly release our annotations, code, and trained models at: github.com/akgulburak/gaze-estimation-fairness