Visibility Allocation Systems: How Algorithmic Design Shapes Online Visibility and Societal Outcomes

TL;DR

This paper introduces a formal framework for visibility allocation systems (VASs) that decide what data to show users, helping evaluate, understand, and regulate complex algorithmic systems affecting societal outcomes.

Contribution

It provides a comprehensive formal framework for analyzing VASs, including tools, metrics, and case studies, to improve understanding and regulation of algorithmic visibility systems.

Findings

Decomposes VASs into sub-processes with data flow diagrams

Surveys metrics for evaluating VASs throughout the pipeline

Demonstrates framework application using school choice forecasting

Abstract

Throughout application domains, we now rely extensively on algorithmic systems to engage with ever-expanding datasets of information. Despite their benefits, these systems are often complex (comprising of many intricate tools, e.g., moderation, recommender systems, prediction models), of unknown structure (due to the lack of accompanying documentation), and having hard-to-predict yet potentially severe downstream consequences (due to the extensive use, systematic enactment of existing errors, and many comprising feedback loops). As such, understanding and evaluating these systems as a whole remains a challenge for both researchers and legislators. To aid ongoing efforts, we introduce a formal framework for such visibility allocation systems (VASs) which we define as (semi-)automated systems deciding which (processed) data to present a human user with. We review typical tools comprising VASs and define the associated computational problems they solve. By doing so, VASs can be decomposed into sub-processes and illustrated via data flow diagrams. Moreover, we survey metrics for evaluating VASs throughout the pipeline, thus aiding system diagnostics. Using forecasting-based recommendations in school choice as a case study, we demonstrate how our framework can support VAS evaluation. We also discuss how our framework can support ongoing AI-legislative efforts to locate obligations, quantify systemic risks, and enable adaptive compliance.