DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Texture Generation on 3D Meshes

TL;DR

This paper introduces DoubleDiffusion, a novel method combining heat dissipation diffusion with denoising diffusion to directly generate textures on 3D meshes, improving efficiency and addressing issues of geometric inconsistencies in existing multi-view image-based approaches.

Contribution

The paper presents a new approach that integrates heat dissipation diffusion with denoising diffusion for native 3D mesh texture generation, enhancing efficiency and accuracy.

Findings

Significantly improved texture generation efficiency.

Reduced geometric inconsistencies and artifacts.

Effective native diffusion-based texture synthesis on 3D meshes.

Abstract

This paper addresses the problem of generating textures for 3D mesh assets. Existing approaches often rely on image diffusion models to generate multi-view image observations, which are then transformed onto the mesh surface to produce a single texture. However, due to the gap between multi-view images and 3D space, such process is susceptible to arange of issues such as geometric inconsistencies, visibility occlusion, and baking artifacts. To overcome this problem, we propose a novel approach that directly generates texture on 3D meshes. Our approach leverages heat dissipation diffusion, which serves as an efficient operator that propagates features on the geometric surface of a mesh, while remaining insensitive to the specific layout of the wireframe. By integrating this technique into a generative diffusion pipeline, we significantly improve the efficiency of texture generation compared to existing texture generation methods. We term our approach DoubleDiffusion, as it combines heat dissipation diffusion with denoising diffusion to enable native generative learning on 3D mesh surfaces.