Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Models

TL;DR

This paper introduces Differential Vector Erasure (DVE), a training-free method for concept erasure in flow matching models, effectively removing undesirable concepts from generated images without fine-tuning.

Contribution

DVE is the first concept erasure approach tailored for flow matching models, leveraging the velocity field's directional structure for precise concept removal.

Findings

DVE outperforms existing methods on NSFW suppression, style removal, and object erasure.

DVE preserves image quality and diversity during concept erasure.

DVE is training-free and applicable during inference.

Abstract

Text-to-image diffusion models have demonstrated remarkable capabilities in generating high-quality images, yet their tendency to reproduce undesirable concepts, such as NSFW content, copyrighted styles, or specific objects, poses growing concerns for safe and controllable deployment. While existing concept erasure approaches primarily focus on DDPM-based diffusion models and rely on costly fine-tuning, the recent emergence of flow matching models introduces a fundamentally different generative paradigm for which prior methods are not directly applicable. In this paper, we propose Differential Vector Erasure (DVE), a training-free concept erasure method specifically designed for flow matching models. Our key insight is that semantic concepts are implicitly encoded in the directional structure of the velocity field governing the generative flow. Leveraging this observation, we construct a differential vector field that characterizes the directional discrepancy between a target concept and a carefully chosen anchor concept. During inference, DVE selectively removes concept-specific components by projecting the velocity field onto the differential direction, enabling precise concept suppression without affecting irrelevant semantics. Extensive experiments on FLUX demonstrate that DVE consistently outperforms existing baselines on a wide range of concept erasure tasks, including NSFW suppression, artistic style removal, and object erasure, while preserving image quality and diversity.