StreamingEffect: Real-Time Human-Centric Video Effect Generation

TL;DR

StreamingEffect introduces a real-time, human-centric video effect generation framework that combines in-context editing, a large curated dataset, and efficient distillation to enable high-quality live video editing.

Contribution

The paper presents a novel real-time streaming video effect framework with a new in-context editing architecture, a large dataset, and effective model distillation techniques.

Findings

Enables real-time 720p video editing on a single GPU.

Introduces keyframe control for interactive online effect injection.

Constructs VideoEffect-130K, the largest human-centric video effect dataset.

Abstract

Streaming video effect generation is highly desirable for live human-centric applications such as e-commerce streaming, entertainment, and vlogging, yet remains difficult due to the lack of suitable data and deployable editing models. Unlike generic video generation, this task requires real-time video-to-video editing that adds expressive effects while preserving human identity, background content, and temporal consistency. Existing acceleration efforts mainly focus on text-to-video generation, while efficient distillation for video editing remains largely underexplored. In this paper, we present \textbf{StreamingEffect}, a real-time human-centric streaming video effect framework. We adopt an in-context video editing architecture and train a high-quality bidirectional teacher, then distill it into a causal autoregressive student and further reduce sampling from 50 steps to 4 steps. We also introduce keyframe control, allowing reference effect frames to be injected online and propagated through the stream for interactive editing. To address the data bottleneck, we construct \textbf{VideoEffect-130K}, to our knowledge the largest human-centric video effect dataset, containing 70K effect videos and 60K editing videos across 600 effect categories curated from short-video and editing platforms. Experiments show that our method enables real-time, high-quality 720p video editing on a single H200 GPU.