Rotate Your Character: Revisiting Video Diffusion Models for High-Quality 3D Character Generation

TL;DR

This paper introduces RCM, a diffusion-based framework for high-quality 3D character generation from images, enabling consistent view synthesis, high resolution, and multi-view conditioning, advancing digital content creation.

Contribution

RCM is a novel diffusion framework that achieves pose transfer, high-resolution view synthesis, and multi-view conditioning for 3D character generation from images.

Findings

Outperforms state-of-the-art in view synthesis quality

Supports high-resolution 1024x1024 video generation

Enables multi-view conditioning with up to 4 images

Abstract

Generating high-quality 3D characters from single images remains a significant challenge in digital content creation, particularly due to complex body poses and self-occlusion. In this paper, we present RCM (Rotate your Character Model), an advanced image-to-video diffusion framework tailored for high-quality novel view synthesis (NVS) and 3D character generation. Compared to existing diffusion-based approaches, RCM offers several key advantages: (1) transferring characters with any complex poses into a canonical pose, enabling consistent novel view synthesis across the entire viewing orbit, (2) high-resolution orbital video generation at 1024x1024 resolution, (3) controllable observation positions given different initial camera poses, and (4) multi-view conditioning supporting up to 4 input images, accommodating diverse user scenarios. Extensive experiments demonstrate that RCM outperforms state-of-the-art methods in both novel view synthesis and 3D generation quality.