SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation

TL;DR

SymphoMotion is a unified framework for controlling camera motion and object dynamics simultaneously, improving coherence and realism in video generation.

Contribution

It introduces a joint control model with geometry-aware cues and a new dataset, addressing limitations of existing methods that handle only one motion type.

Findings

Outperforms existing methods in visual fidelity and controllability.

Enables depth-aware, spatially coherent object manipulation.

Provides a new dataset for unified motion control in videos.

Abstract

Controlling both camera motion and object dynamics is essential for coherent and expressive video generation, yet current methods typically handle only one motion type or rely on ambiguous 2D cues that entangle camera-induced parallax with true object movement. We present SymphoMotion, a unified motion-control framework that jointly governs camera trajectories and object dynamics within a single model. SymphoMotion features a Camera Trajectory Control mechanism that integrates explicit camera paths with geometry-aware cues to ensure stable, structurally consistent viewpoint transitions, and an Object Dynamics Control mechanism that combines 2D visual guidance with 3D trajectory embeddings to enable depth-aware, spatially coherent object manipulation. To support large-scale training and evaluation, we further construct RealCOD-25K, a comprehensive real-world dataset containing paired camera poses and object-level 3D trajectories across diverse indoor and outdoor scenes, addressing a key data gap in unified motion control. Extensive experiments and user studies show that SymphoMotion significantly outperforms existing methods in visual fidelity, camera controllability, and object-motion accuracy, establishing a new benchmark for unified motion control in video generation. Codes and data are publicly available at https://grenoble-zhang.github.io/SymphoMotion/.