LensCraft: Your Professional Virtual Cinematographer

TL;DR

LensCraft is a data-driven, real-time virtual cinematographer system that accurately translates creative vision into camera movements, considering subject volume and dynamics, outperforming existing models in accuracy and coherence.

Contribution

It introduces a novel neural model combining cinematographic principles with dynamic scene awareness, enabling flexible, high-fidelity camera control in real time.

Findings

Achieves higher accuracy and coherence than state-of-the-art models.

Supports multiple input modalities for versatile control.

Maintains low computational complexity with fast inference.

Abstract

Digital creators, from indie filmmakers to animation studios, face a persistent bottleneck: translating their creative vision into precise camera movements. Despite significant progress in computer vision and artificial intelligence, current automated filming systems struggle with a fundamental trade-off between mechanical execution and creative intent. Crucially, almost all previous works simplify the subject to a single point-ignoring its orientation and true volume-severely limiting spatial awareness during filming. LensCraft solves this problem by mimicking the expertise of a professional cinematographer, using a data-driven approach that combines cinematographic principles with the flexibility to adapt to dynamic scenes in real time. Our solution combines a specialized simulation framework for generating high-fidelity training data with an advanced neural model that is faithful to the script while being aware of the volume and dynamic behavior of the subject. Additionally, our approach allows for flexible control via various input modalities, including text prompts, subject trajectory and volume, key points, or a full camera trajectory, offering creators a versatile tool to guide camera movements in line with their vision. Leveraging a lightweight real time architecture, LensCraft achieves markedly lower computational complexity and faster inference while maintaining high output quality. Extensive evaluation across static and dynamic scenarios reveals unprecedented accuracy and coherence, setting a new benchmark for intelligent camera systems compared to state-of-the-art models. Extended results, the complete dataset, simulation environment, trained model weights, and source code are publicly accessible on LensCraft Webpage.