HECTOR: Hybrid Editable Compositional Object References for Video Generation

TL;DR

HECTOR introduces a novel generative pipeline for video synthesis that allows explicit, fine-grained compositional control using hybrid references and trajectory specifications, enhancing coherence and fidelity.

Contribution

It presents a new method supporting hybrid reference conditioning and explicit trajectory control, enabling more precise and controllable video generation.

Findings

Superior visual quality over existing methods

Enhanced reference preservation in generated videos

Improved motion controllability and coherence

Abstract

Real-world videos naturally portray complex interactions among distinct physical objects, effectively forming dynamic compositions of visual elements. However, most current video generation models synthesize scenes holistically and therefore lack mechanisms for explicit compositional manipulation. To address this limitation, we propose HECTOR, a generative pipeline that enables fine-grained compositional control. In contrast to prior methods,HECTOR supports hybrid reference conditioning, allowing generation to be simultaneously guided by static images and/or dynamic videos. Moreover, users can explicitly specify the trajectory of each referenced element, precisely controlling its location, scale, and speed (see Figure1). This design allows the model to synthesize coherent videos that satisfy complex spatiotemporal constraints while preserving high-fidelity adherence to references. Extensive experiments demonstrate that HECTOR achieves superior visual quality, stronger reference preservation, and improved motion controllability compared with existing approaches.