PrivLEX: Detecting legal concepts in images through Vision-Language Models

TL;DR

PrivLEX is an innovative privacy classifier that uses vision-language models to identify legally defined personal data concepts in images, providing interpretability without needing explicit concept labels during training.

Contribution

It introduces PrivLEX, the first interpretable privacy classifier grounded in legal concepts and leveraging zero-shot vision-language models for concept detection.

Findings

PrivLEX effectively identifies personal data concepts in images.

PrivLEX offers interpretability aligned with legal privacy standards.

Analysis shows human perception of privacy sensitivity varies with concepts.

Abstract

We present PrivLEX, a novel image privacy classifier that grounds its decisions in legally defined personal data concepts. PrivLEX is the first interpretable privacy classifier aligned with legal concepts that leverages the recognition capabilities of Vision-Language Models (VLMs). PrivLEX relies on zero-shot VLM concept detection to provide interpretable classification through a label-free Concept Bottleneck Model, without requiring explicit concept labels during training. We demonstrate PrivLEX's ability to identify personal data concepts that are present in images. We further analyse the sensitivity of such concepts as perceived by human annotators of image privacy datasets.