RISE: Interactive Visual Diagnosis of Fairness in Machine Learning Models

TL;DR

RISE is an interactive visualization tool that helps diagnose fairness issues in machine learning models by revealing detailed disparity patterns and trade-offs that aggregate metrics often hide.

Contribution

It introduces RISE, a novel visualization method that connects residual patterns to fairness notions for localized and comparative fairness analysis.

Findings

Enables detection of hidden fairness issues.

Reveals accuracy-fairness trade-offs.

Supports informed model selection.

Abstract

Evaluating fairness under domain shift is challenging because scalar metrics often obscure exactly where and how disparities arise. We introduce \textit{RISE} (Residual Inspection through Sorted Evaluation), an interactive visualization tool that converts sorted residuals into interpretable patterns. By connecting residual curve structures to formal fairness notions, RISE enables localized disparity diagnosis, subgroup comparison across environments, and the detection of hidden fairness issues. Through post-hoc analysis, RISE exposes accuracy-fairness trade-offs that aggregate statistics miss, supporting more informed model selection.