Linting is People! Exploring the Potential of Human Computation as a Sociotechnical Linter of Data Visualizations

TL;DR

This paper investigates using human computation as a sociotechnical approach to improve the evaluation and critique of data visualizations, extending traditional linting methods with social and contextual insights.

Contribution

It introduces a novel framework for leveraging crowd-sourced human judgment in visualization linting, enhancing detection of misleading content and integrating social perspectives.

Findings

Human assessments identify misleading visualizations effectively.

Crowd-sourced critiques add valuable social context.

Integrating human computation improves traditional linting methods.

Abstract

Traditionally, linters are code analysis tools that help developers by flagging potential issues from syntax and logic errors to enforcing syntactical and stylistic conventions. Recently, linting has been taken as an interface metaphor, allowing it to be extended to more complex inputs, such as visualizations, which demand a broader perspective and alternative approach to evaluation. We explore a further extended consideration of linting inputs, and modes of evaluation, across the puritanical, neutral, and rebellious dimensions. We specifically investigate the potential for leveraging human computation in linting operations through Community Notes -- crowd-sourced contextual text snippets aimed at checking and critiquing potentially accurate or misleading content on social media. We demonstrate that human-powered assessments not only identify misleading or error-prone visualizations but that integrating human computation enhances traditional linting by offering social insights. As is required these days, we consider the implications of building linters powered by Artificial Intelligence.