OneTo3D: One Image to Re-editable Dynamic 3D Model and Video Generation

TL;DR

OneTo3D introduces a novel method for converting a single image into an editable 3D model and a continuous 3D video, leveraging Gaussian Splatting and adaptive mechanisms for precise motion control and semantic consistency.

Contribution

The paper presents a new approach combining Gaussian Splatting with adaptive binding and motion control algorithms to generate editable 3D models and unlimited 3D videos from a single image.

Findings

Outperforms state-of-the-art in motion and action control accuracy.

Enables generation of long, semantic, continuous 3D videos.

Uses less computational resources compared to existing methods.

Abstract

One image to editable dynamic 3D model and video generation is novel direction and change in the research area of single image to 3D representation or 3D reconstruction of image. Gaussian Splatting has demonstrated its advantages in implicit 3D reconstruction, compared with the original Neural Radiance Fields. As the rapid development of technologies and principles, people tried to used the Stable Diffusion models to generate targeted models with text instructions. However, using the normal implicit machine learning methods is hard to gain the precise motions and actions control, further more, it is difficult to generate a long content and semantic continuous 3D video. To address this issue, we propose the OneTo3D, a method and theory to used one single image to generate the editable 3D model and generate the targeted semantic continuous time-unlimited 3D video. We used a normal basic Gaussian Splatting model to generate the 3D model from a single image, which requires less volume of video memory and computer calculation ability. Subsequently, we designed an automatic generation and self-adaptive binding mechanism for the object armature. Combined with the re-editable motions and actions analyzing and controlling algorithm we proposed, we can achieve a better performance than the SOTA projects in the area of building the 3D model precise motions and actions control, and generating a stable semantic continuous time-unlimited 3D video with the input text instructions. Here we will analyze the detailed implementation methods and theories analyses. Relative comparisons and conclusions will be presented. The project code is open source.