Example-Based Feature Painting on Textures

TL;DR

This paper presents a learning-based, unsupervised system for controlled editing and generation of textures with local blemishes, enabling realistic and interactive texture creation without manual annotations.

Contribution

It introduces a novel unsupervised approach for texture editing that clusters features and guides conditional generation, including diffusion-based editing and infinite texture synthesis.

Findings

Effective unsupervised anomaly detection for blemish features

Clustering of textural features into semantic groups

Interactive texture editing and infinite generation capabilities

Abstract

In this work, we propose a system that covers the complete workflow for achieving controlled authoring and editing of textures that present distinctive local characteristics. These include various effects that change the surface appearance of materials, such as stains, tears, holes, abrasions, discoloration, and more. Such alterations are ubiquitous in nature, and including them in the synthesis process is crucial for generating realistic textures. We introduce a novel approach for creating textures with such blemishes, adopting a learning-based approach that leverages unlabeled examples. Our approach does not require manual annotations by the user; instead, it detects the appearance-altering features through unsupervised anomaly detection. The various textural features are then automatically clustered into semantically coherent groups, which are used to guide the conditional generation of images. Our pipeline as a whole goes from a small image collection to a versatile generative model that enables the user to interactively create and paint features on textures of arbitrary size. Notably, the algorithms we introduce for diffusion-based editing and infinite stationary texture generation are generic and should prove useful in other contexts as well. Project page: https://reality.tf.fau.de/pub/ardelean2025examplebased.html