Unsupervised Cross-Domain Regression for Fine-grained 3D Game Character Reconstruction

TL;DR

This paper introduces an unsupervised, cross-domain framework for reconstructing detailed 3D game characters from single images, effectively bridging the gap between real-world and game domains with novel loss functions and identity-aware features.

Contribution

The paper presents a new end-to-end unsupervised method that reduces domain discrepancy and preserves individual details in 3D character reconstruction from single images.

Findings

Achieves state-of-the-art results in 3D game character reconstruction.

Effectively transfers knowledge across domains without ground truth data.

Maintains person-specific details through contrastive loss.

Abstract

With the rise of the ``metaverse'' and the rapid development of games, it has become more and more critical to reconstruct characters in the virtual world faithfully. The immersive experience is one of the most central themes of the ``metaverse'', while the reducibility of the avatar is the crucial point. Meanwhile, the game is the carrier of the metaverse, in which players can freely edit the facial appearance of the game character. In this paper, we propose a simple but powerful cross-domain framework that can reconstruct fine-grained 3D game characters from single-view images in an end-to-end manner. Different from the previous methods, which do not resolve the cross-domain gap, we propose an effective regressor that can greatly reduce the discrepancy between the real-world domain and the game domain. To figure out the drawbacks of no ground truth, our unsupervised framework has accomplished the knowledge transfer of the target domain. Additionally, an innovative contrastive loss is proposed to solve the instance-wise disparity, which keeps the person-specific details of the reconstructed character. In contrast, an auxiliary 3D identity-aware extractor is activated to make the results of our model more impeccable. Then a large set of physically meaningful facial parameters is generated robustly and exquisitely. Experiments demonstrate that our method yields state-of-the-art performance in 3D game character reconstruction.