"I inherently just trust that it works": Investigating Mental Models of Open-Source Libraries for Differential Privacy

TL;DR

This paper investigates how open-source differential privacy libraries influence user understanding and trust, revealing gaps between developer designs and user mental models, and offers recommendations for improving library design.

Contribution

It provides a qualitative analysis of the disconnect between developer and user mental models in DP libraries, highlighting design challenges and proposing practical improvements.

Findings

DP libraries often fail to align developer and user mental models

There is tension between maintaining rigorous DP and facilitating user interaction

Recommendations for improving DP library design to bridge understanding gaps

Abstract

Differential privacy (DP) is a promising framework for privacy-preserving data science, but recent studies have exposed challenges in bringing this theoretical framework for privacy into practice. These tensions are particularly salient in the context of open-source software libraries for DP data analysis, which are emerging tools to help data stewards and analysts build privacy-preserving data pipelines for their applications. While there has been significant investment into such libraries, we need further inquiry into the role of these libraries in promoting understanding of and trust in DP, and in turn, the ways in which design of these open-source libraries can shed light on the challenges of creating trustworthy data infrastructures in practice. In this study, we use qualitative methods and mental models approaches to analyze the differences between conceptual models used to design open-source DP libraries and mental models of DP held by users. Through a two-stage study design involving formative interviews with 5 developers of open-source DP libraries and user studies with 17 data analysts, we find that DP libraries often struggle to bridge the gaps between developer and user mental models. In particular, we highlight the tension DP libraries face in maintaining rigorous DP implementations and facilitating user interaction. We conclude by offering practical recommendations for further development of DP libraries.