Automatic Camera Trajectory Control with Enhanced Immersion for Virtual Cinematography
Xinyi Wu, Haohong Wang, Aggelos K. Katsaggelos

TL;DR
This paper introduces a deep learning framework for automatic camera control in virtual cinematography that synchronizes camera movements with actor actions and emotions to produce immersive, high-quality cinematic videos.
Contribution
It presents a novel actor-camera synchronization method considering aesthetics, physical action, and emotional state, using a GAN-based model and self-supervised adjustments.
Findings
Produces immersive cinematic videos with high quality
Effectively synchronizes camera with actor’s actions and emotions
Outperforms existing methods in qualitative and quantitative evaluations
Abstract
User-generated cinematic creations are gaining popularity as our daily entertainment, yet it is a challenge to master cinematography for producing immersive contents. Many existing automatic methods focus on roughly controlling predefined shot types or movement patterns, which struggle to engage viewers with the circumstances of the actor. Real-world cinematographic rules show that directors can create immersion by comprehensively synchronizing the camera with the actor. Inspired by this strategy, we propose a deep camera control framework that enables actor-camera synchronization in three aspects, considering frame aesthetics, spatial action, and emotional status in the 3D virtual stage. Following rule-of-thirds, our framework first modifies the initial camera placement to position the actor aesthetically. This adjustment is facilitated by a self-supervised adjustor that analyzes frame…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Advanced Image Processing Techniques
