Technocracy, pseudoscience and performative compliance: the risks of privacy risk assessments. Lessons from NIST's Privacy Risk Assessment Methodology

TL;DR

This paper critically examines NIST's Privacy Risk Assessment Methodology, revealing its vulnerabilities to performative compliance and questioning its effectiveness in genuinely protecting privacy within a regulatory framework.

Contribution

The paper provides a detailed analysis of NIST's privacy risk assessment approach, exposing its limitations and proposing the need for alternative strategies and policy realignment.

Findings

Opportunities for adversarial organizations to engage in performative compliance.

Limitations in regulators' auditing capabilities.

Challenges to the assumptions underpinning privacy risk assessments.

Abstract

Privacy risk assessments have been touted as an objective, principled way to encourage organizations to implement privacy-by-design. They are central to a new regulatory model of collaborative governance, as embodied by the GDPR. However, existing guidelines and methods remain vague, and there is little empirical evidence on privacy harms. In this paper we conduct a close analysis of US NIST's Privacy Risk Assessment Methodology, highlighting multiple sites of discretion that create countless opportunities for adversarial organizations to engage in performative compliance. Our analysis shows that the premises on which the success of privacy risk assessments depends do not hold, particularly in regard to organizations' incentives and regulators auditing capabilities. We highlight the limitations and pitfalls of what is essentially a utilitarian and technocratic approach, leading us to discuss alternatives and a realignment of our policy and research objectives.