Over++: Generative Video Compositing for Layer Interaction Effects

TL;DR

Over++ is a novel framework for generating realistic environmental effects in videos, conditioned on text prompts, that preserves original scenes without requiring extensive annotations or scene assumptions.

Contribution

We introduce augmented compositing and Over++, a new video effect generation method that handles complex environmental effects with minimal supervision and flexible control options.

Findings

Over++ outperforms existing methods in effect realism and scene preservation.

The dataset and augmentation strategy enable effective training with limited data.

Over++ supports mask control and keyframe guidance for flexible editing.

Abstract

In professional video compositing workflows, artists must manually create environmental interactions-such as shadows, reflections, dust, and splashes-between foreground subjects and background layers. Existing video generative models struggle to preserve the input video while adding such effects, and current video inpainting methods either require costly per-frame masks or yield implausible results. We introduce augmented compositing, a new task that synthesizes realistic, semi-transparent environmental effects conditioned on text prompts and input video layers, while preserving the original scene. To address this task, we present Over++, a video effect generation framework that makes no assumptions about camera pose, scene stationarity, or depth supervision. We construct a paired effect dataset tailored for this task and introduce an unpaired augmentation strategy that preserves text-driven editability. Our method also supports optional mask control and keyframe guidance without requiring dense annotations. Despite training on limited data, Over++ produces diverse and realistic environmental effects and outperforms existing baselines in both effect generation and scene preservation.