Controlling Material Appearance by Examples

TL;DR

This paper introduces an example-based method for editing and controlling material appearance in rendering by augmenting material maps with user photos, leveraging a trained MaterialGAN and differentiable rendering.

Contribution

It presents a novel approach that uses MaterialGAN's prior and differentiable rendering to transfer textures from user photos onto material maps, enhancing realism and control.

Findings

Effective transfer of micro and meso-structure textures

Improved realism in material map editing

Ability to generate new visually appealing materials

Abstract

Despite the ubiquitousness of materials maps in modern rendering pipelines, their editing and control remains a challenge. In this paper, we present an example-based material control method to augment input material maps based on user-provided material photos. We train a tileable version of MaterialGAN and leverage its material prior to guide the appearance transfer, optimizing its latent space using differentiable rendering. Our method transfers the micro and meso-structure textures of user provided target(s) photographs, while preserving the structure of the input and quality of the input material. We show our methods can control existing material maps, increasing realism or generating new, visually appealing materials.