GreaseVision: Rewriting the Rules of the Interface

TL;DR

GreaseVision is a no-code framework that empowers end-users to collaboratively identify and create personalized interventions against digital harms, leveraging recent machine learning advances for scalable harm analysis.

Contribution

It introduces a novel framework and tool that allow end-users to study their digital harms and develop interventions without coding, enabling scalable research and personalized solutions.

Findings

End-users can effectively create personalized interventions.

The framework facilitates large-scale study of harms and interventions.

It leverages few-shot machine learning for adaptive harm mitigation.

Abstract

Digital harms can manifest across any interface. Key problems in addressing these harms include the high individuality of harms and the fast-changing nature of digital systems. As a result, we still lack a systematic approach to study harms and produce interventions for end-users. We put forward GreaseVision, a new framework that enables end-users to collaboratively develop interventions against harms in software using a no-code approach and recent advances in few-shot machine learning. The contribution of the framework and tool allow individual end-users to study their usage history and create personalized interventions. Our contribution also enables researchers to study the distribution of harms and interventions at scale.