Harnessing Hyperbolic Geometry for Harmful Prompt Detection and Sanitization

TL;DR

This paper introduces a novel framework using hyperbolic geometry to detect and sanitize harmful prompts in vision-language models, enhancing safety and robustness.

Contribution

It presents HyPE and HyPS, two hyperbolic geometry-based components for anomaly detection and explainable sanitization of malicious prompts in VLMs.

Findings

Outperforms prior defenses in detection accuracy.

Demonstrates robustness against embedding-level attacks.

Provides an interpretable approach to prompt sanitization.

Abstract

Vision-Language Models (VLMs) have become essential for tasks such as image synthesis, captioning, and retrieval by aligning textual and visual information in a shared embedding space. Yet, this flexibility also makes them vulnerable to malicious prompts designed to produce unsafe content, raising critical safety concerns. Existing defenses either rely on blacklist filters, which are easily circumvented, or on heavy classifier-based systems, both of which are costly and fragile under embedding-level attacks. We address these challenges with two complementary components: Hyperbolic Prompt Espial (HyPE) and Hyperbolic Prompt Sanitization (HyPS). HyPE is a lightweight anomaly detector that leverages the structured geometry of hyperbolic space to model benign prompts and detect harmful ones as outliers. HyPS builds on this detection by applying explainable attribution methods to identify and selectively modify harmful words, neutralizing unsafe intent while preserving the original semantics of user prompts. Through extensive experiments across multiple datasets and adversarial scenarios, we prove that our framework consistently outperforms prior defenses in both detection accuracy and robustness. Together, HyPE and HyPS offer an efficient, interpretable, and resilient approach to safeguarding VLMs against malicious prompt misuse.