Stream-Based Monitoring of Algorithmic Fairness
Jan Baumeister, Bernd Finkbeiner, Frederik Scheerer, Julian Siber,, Tobias Wagenpfeil

TL;DR
This paper introduces a stream-based runtime monitoring approach for verifying algorithmic fairness in decision systems, capable of handling complex, large-scale, real-world data streams.
Contribution
It formalizes algorithmic fairness over data streams using RTLola and demonstrates its effectiveness on synthetic benchmarks and real-world recidivism data.
Findings
Effective at monitoring fairness in large-scale data streams
Successfully applied to real-world COMPAS recidivism data
Outperforms static verification methods in complex scenarios
Abstract
Automatic decision and prediction systems are increasingly deployed in applications where they significantly impact the livelihood of people, such as for predicting the creditworthiness of loan applicants or the recidivism risk of defendants. These applications have given rise to a new class of algorithmic-fairness specifications that require the systems to decide and predict without bias against social groups. Verifying these specifications statically is often out of reach for realistic systems, since the systems may, e.g., employ complex learning components, and reason over a large input space. In this paper, we therefore propose stream-based monitoring as a solution for verifying the algorithmic fairness of decision and prediction systems at runtime. Concretely, we present a principled way to formalize algorithmic fairness over temporal data streams in the specification language…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBlockchain Technology Applications and Security · IoT and Edge/Fog Computing · Ethics and Social Impacts of AI
