ConceptGuard: Proactive Safety in Text-and-Image-to-Video Generation through Multimodal Risk Detection

TL;DR

ConceptGuard is a proactive safety framework for multimodal video generation that detects and mitigates unsafe content by analyzing fused image-text inputs and intervening during the generation process.

Contribution

It introduces a novel two-stage safety mechanism and two benchmarks for evaluating safety in text-and-image-to-video generation models.

Findings

Outperforms existing safety baselines in risk detection.

Achieves state-of-the-art safe video generation results.

Provides large-scale datasets for multimodal safety evaluation.

Abstract

Recent progress in video generative models has enabled the creation of high-quality videos from multimodal prompts that combine text and images. While these systems offer enhanced controllability, they also introduce new safety risks, as harmful content can emerge from individual modalities or their interaction. Existing safety methods are often text-only, require prior knowledge of the risk category, or operate as post-generation auditors, struggling to proactively mitigate such compositional, multimodal risks. To address this challenge, we present ConceptGuard, a unified safeguard framework for proactively detecting and mitigating unsafe semantics in multimodal video generation. ConceptGuard operates in two stages: First, a contrastive detection module identifies latent safety risks by projecting fused image-text inputs into a structured concept space; Second, a semantic suppression mechanism steers the generative process away from unsafe concepts by intervening in the prompt's multimodal conditioning. To support the development and rigorous evaluation of this framework, we introduce two novel benchmarks: ConceptRisk, a large-scale dataset for training on multimodal risks, and T2VSafetyBench-TI2V, the first benchmark adapted from T2VSafetyBench for the Text-and-Image-to-Video (TI2V) safety setting. Comprehensive experiments on both benchmarks show that ConceptGuard consistently outperforms existing baselines, achieving state-of-the-art results in both risk detection and safe video generation. Our code is available at https://github.com/Ruize-Ma/ConceptGuard.