Texture Generation Using Dual-Domain Feature Flow with Multi-View Hallucinations

TL;DR

This paper introduces a dual-domain generative model that estimates complete, consistent texture maps from a single image by generating multi-view hallucinations and exchanging features via a flow-based attention mechanism, improving 3D rendering and pose transfer.

Contribution

The novel dual-domain model simultaneously generates multi-view hallucinations and texture maps, utilizing feature exchange to enhance texture estimation from a single image.

Findings

Outperforms existing methods in texture map estimation accuracy.

Produces more consistent and complete textures for 3D models.

Enhances image generation quality for pose and viewpoint transfer.

Abstract

We propose a dual-domain generative model to estimate a texture map from a single image for colorizing a 3D human model. When estimating a texture map, a single image is insufficient as it reveals only one facet of a 3D object. To provide sufficient information for estimating a complete texture map, the proposed model simultaneously generates multi-view hallucinations in the image domain and an estimated texture map in the texture domain. During the generating process, each domain generator exchanges features to the other by a flow-based local attention mechanism. In this manner, the proposed model can estimate a texture map utilizing abundant multi-view image features from which multiview hallucinations are generated. As a result, the estimated texture map contains consistent colors and patterns over the entire region. Experiments show the superiority of our model for estimating a directly render-able texture map, which is applicable to 3D animation rendering. Furthermore, our model also improves an overall generation quality in the image domain for pose and viewpoint transfer tasks.