Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs

TL;DR

This paper proposes a new compositional framework for visual privacy risk assessment that considers attribute combinations, introduces a severity scoring system, and evaluates model performance on privacy severity prediction.

Contribution

It introduces the Compositional Privacy Risk Taxonomy (CPRT), a graded severity framework, and a dataset for assessing privacy risks with vision-language models, highlighting model limitations.

Findings

Frontier models align with severity when guided but underestimate risks.

Smaller models struggle with graded privacy reasoning.

An 8B fine-tuned model matches frontier performance on privacy assessment.

Abstract

Existing visual privacy benchmarks largely treat privacy as a binary property, labeling images as private or non-private based on visible sensitive content. We argue that privacy is fundamentally compositional. Attributes that are benign in isolation may combine to produce severe privacy violations. We introduce the Compositional Privacy Risk Taxonomy (CPRT), a regulation-aware framework that organizes visual attributes according to standalone identifiability and compositional harm potential. CPRT defines four graded severity levels and is paired with an interpretable scoring function that assigns continuous privacy severity scores. We further construct a taxonomy-aligned dataset of 6.7K images and derive ground-truth compositional risk scores. By evaluating frontier and open-weight VLMs we find that frontier models align well with compositional severity when provided structured guidance, but systematically underestimate composition-driven risks. Smaller models struggle to internalize graded privacy reasoning. To bridge this gap, we introduce a deployable 8B supervised fine-tuned (SFT) model that closely matches frontier-level performance on compositional privacy assessment.