PV-VTT: A Privacy-Centric Dataset for Mission-Specific Anomaly Detection and Natural Language Interpretation

TL;DR

This paper introduces PV-VTT, a privacy-preserving multimodal dataset for detecting and describing privacy violations in videos, along with a GNN-based model that enhances natural language interpretation while safeguarding individual privacy.

Contribution

The paper presents PV-VTT, a novel privacy-centric dataset with detailed annotations and a GNN-based video description model that reduces token usage for LLMs.

Findings

PV-VTT effectively detects privacy violations with detailed annotations.

The GNN-based model generates high-quality descriptions using minimal input.

Experiments show the approach's effectiveness and interpretability.

Abstract

Video crime detection is a significant application of computer vision and artificial intelligence. However, existing datasets primarily focus on detecting severe crimes by analyzing entire video clips, often neglecting the precursor activities (i.e., privacy violations) that could potentially prevent these crimes. To address this limitation, we present PV-VTT (Privacy Violation Video To Text), a unique multimodal dataset aimed at identifying privacy violations. PV-VTT provides detailed annotations for both video and text in scenarios. To ensure the privacy of individuals in the videos, we only provide video feature vectors, avoiding the release of any raw video data. This privacy-focused approach allows researchers to use the dataset while protecting participant confidentiality. Recognizing that privacy violations are often ambiguous and context-dependent, we propose a Graph Neural Network (GNN)-based video description model. Our model generates a GNN-based prompt with image for Large Language Model (LLM), which deliver cost-effective and high-quality video descriptions. By leveraging a single video frame along with relevant text, our method reduces the number of input tokens required, maintaining descriptive quality while optimizing LLM API-usage. Extensive experiments validate the effectiveness and interpretability of our approach in video description tasks and flexibility of our PV-VTT dataset.