Style-Aware Gloss Control for Generative Non-Photorealistic Rendering

TL;DR

This paper presents a style-aware generative model that disentangles gloss from artistic style, enabling detailed control over material appearance in non-photorealistic rendering.

Contribution

We introduce a hierarchical latent space that separates gloss from style and develop an adapter to control these factors in diffusion-based image synthesis.

Findings

Hierarchical latent space effectively disentangles gloss and style.

Our adapter improves controllability of gloss and style in generated images.

Enhanced disentanglement compared to previous models.

Abstract

Humans can infer material characteristics of objects from their visual appearance, and this ability extends to artistic depictions, where similar perceptual strategies guide the interpretation of paintings or drawings. Among the factors that define material appearance, gloss, along with color, is widely regarded as one of the most important, and recent studies indicate that humans can perceive gloss independently of the artistic style used to depict an object. To investigate how gloss and artistic style are represented in learned models, we train an unsupervised generative model on a newly curated dataset of painterly objects designed to systematically vary such factors. Our analysis reveals a hierarchical latent space in which gloss is disentangled from other appearance factors, allowing for a detailed study of how gloss is represented and varies across artistic styles. Building on this representation, we introduce a lightweight adapter that connects our style- and gloss-aware latent space to a latent-diffusion model, enabling the synthesis of non-photorealistic images with fine-grained control of these factors. We compare our approach with previous models and observe improved disentanglement and controllability of the learned factors.