Human-AI Interaction for User Safety in Social Matching Apps: Involving Marginalized Users in Design

TL;DR

This paper advocates for participatory design in social matching apps to enhance safety for marginalized users, involving them in redesigning AI interactions to prevent harms like sexual violence.

Contribution

It introduces participatory design methods to involve marginalized users in creating safer AI-driven social matching experiences.

Findings

Marginalized users can effectively redesign apps to improve safety.

Participatory design empowers women and LGBTQIA+ users to influence AI safety features.

Prototypes focus on consent mediation and safety-oriented social matching.

Abstract

In this position paper we intend to advocate for participatory design methods and mobile social matching apps as ripe contexts for exploring novel human-AI interactions that benefit marginalized groups. Mobile social matching apps like Tinder and Bumble use AI to introduce users to each other for rapid face-to-face meetings. These user discoveries and subsequent interactions pose disproportionate risk of sexual violence and other harms to marginalized user demographics, specifically women and the LGBTQIA+ community. We want to extend the role of AI in these apps to keep users safe while they interact with strangers across online and offline modalities. To do this, we are using participatory design methods to empower women and LGBTQIA+ individuals to envision future human-AI interactions that prioritize their safety during social matching app-use. In one study, stakeholders identifying as LGBTQIA+ or women are redesigning dating apps to mediate exchange of sexual consent and therefore mitigate sexual violence. In the other study, women are designing multi-purpose, opportunistic social matching apps that foreground women's safety.