Designing AI for Online-to-Offline Safety Risks with Young Women: The Context of Social Matching

TL;DR

This paper explores how AI can help mitigate safety risks faced by young women in online-to-offline social matching platforms, emphasizing participatory design to create explainable risk detection models.

Contribution

It introduces a participatory AI design approach involving young women to develop explainable risk detection models for social matching platforms.

Findings

Early insights from participatory AI design study

Young women contribute to explainable risk models

Highlights challenges in participatory AI methods

Abstract

In this position paper we draw attention to safety risks against youth and young adults that originate through the combination of online and in-person interaction, and opportunities for AI to address these risks. Our context of study is social matching systems (e.g., Tinder, Bumble), which are used by young adults for online-to-offline interaction with strangers, and which are correlated with sexual violence both online and in-person. The paper presents early insights from an ongoing participatory AI design study in which young women build directly explainable models for detecting risk associated with discovered social opportunities, and articulate what AI should do once risk has been detected. We seek to advocate for participatory AI design as a way to directly incorporate youth and young adults into the design of a safer Internet. We also draw attention to challenges with the method.