EASI-Tex: Edge-Aware Mesh Texturing from Single Image

TL;DR

EASI-Tex is a novel method that uses diffusion models conditioned on mesh edges and features to transfer textures from a single image onto 3D meshes without training, effectively preserving details and geometry.

Contribution

The paper introduces EASI-Tex, a new single-image mesh texturing technique that combines diffusion models with edge and feature conditioning, eliminating the need for optimization or training.

Findings

Effective preservation of input texture details on diverse 3D meshes

High efficiency and effectiveness demonstrated through experiments

Able to handle significant geometric and part discrepancies

Abstract

We present a novel approach for single-image mesh texturing, which employs a diffusion model with judicious conditioning to seamlessly transfer an object's texture from a single RGB image to a given 3D mesh object. We do not assume that the two objects belong to the same category, and even if they do, there can be significant discrepancies in their geometry and part proportions. Our method aims to rectify the discrepancies by conditioning a pre-trained Stable Diffusion generator with edges describing the mesh through ControlNet, and features extracted from the input image using IP-Adapter to generate textures that respect the underlying geometry of the mesh and the input texture without any optimization or training. We also introduce Image Inversion, a novel technique to quickly personalize the diffusion model for a single concept using a single image, for cases where the pre-trained IP-Adapter falls short in capturing all the details from the input image faithfully. Experimental results demonstrate the efficiency and effectiveness of our edge-aware single-image mesh texturing approach, coined EASI-Tex, in preserving the details of the input texture on diverse 3D objects, while respecting their geometry.