Usable Differential Privacy: A Case Study with PSI

TL;DR

This paper evaluates the usability of PSI, a tool designed to make differential privacy accessible to non-experts, through a comprehensive user study and analysis of its features.

Contribution

It provides a detailed usability assessment of PSI, offering insights and principles for designing user-friendly differential privacy systems.

Findings

Identified user-friendly features of PSI

Highlighted confusing aspects needing improvement

Derived principles for usable privacy system design

Abstract

Differential privacy is a promising framework for addressing the privacy concerns in sharing sensitive datasets for others to analyze. However differential privacy is a highly technical area and current deployments often require experts to write code, tune parameters, and optimize the trade-off between the privacy and accuracy of statistical releases. For differential privacy to achieve its potential for wide impact, it is important to design usable systems that enable differential privacy to be used by ordinary data owners and analysts. PSI is a tool that was designed for this purpose, allowing researchers to release useful differentially private statistical information about their datasets without being experts in computer science, statistics, or privacy. We conducted a thorough usability study of PSI to test whether it accomplishes its goal of usability by non-experts. The usability test illuminated which features of PSI are most user-friendly and prompted us to improve aspects of the tool that caused confusion. The test also highlighted some general principles and lessons for designing usable systems for differential privacy, which we discuss in depth.