3D Cinemagraphy from a Single Image
Xingyi Li, Zhiguo Cao, Huiqiang Sun, Jianming Zhang, Ke Xian, Guosheng, Lin

TL;DR
This paper introduces 3D Cinemagraphy, a novel technique that creates animated 3D videos from a single image by converting it into a layered depth representation, estimating scene motion, and synthesizing views to produce realistic animations.
Contribution
The paper proposes a new method that combines 2D image animation with 3D scene understanding, addressing artifacts by representing scenes in 3D space from a single image.
Findings
Effective scene representation in 3D space improves animation quality.
Bidirectional point cloud displacement reduces artifacts and holes.
User study confirms the visual quality of generated videos.
Abstract
We present 3D Cinemagraphy, a new technique that marries 2D image animation with 3D photography. Given a single still image as input, our goal is to generate a video that contains both visual content animation and camera motion. We empirically find that naively combining existing 2D image animation and 3D photography methods leads to obvious artifacts or inconsistent animation. Our key insight is that representing and animating the scene in 3D space offers a natural solution to this task. To this end, we first convert the input image into feature-based layered depth images using predicted depth values, followed by unprojecting them to a feature point cloud. To animate the scene, we perform motion estimation and lift the 2D motion into the 3D scene flow. Finally, to resolve the problem of hole emergence as points move forward, we propose to bidirectionally displace the point cloud as per…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Video Analysis and Summarization
