Photorealistic Material Editing Through Direct Image Manipulation

TL;DR

This paper introduces a user-friendly method for creating photorealistic materials by simple image edits, enabling non-experts to generate high-quality results quickly using neural network techniques.

Contribution

It presents a novel workflow combining neural network optimization and encoding to produce photorealistic materials from basic image manipulations, suitable for novice users.

Findings

Produces high-quality results within 30 seconds.

Resilient to poorly-edited target images.

Capable of real-time sequence prediction with 1-2 seconds per image.

Abstract

Creating photorealistic materials for light transport algorithms requires carefully fine-tuning a set of material properties to achieve a desired artistic effect. This is typically a lengthy process that involves a trained artist with specialized knowledge. In this work, we present a technique that aims to empower novice and intermediate-level users to synthesize high-quality photorealistic materials by only requiring basic image processing knowledge. In the proposed workflow, the user starts with an input image and applies a few intuitive transforms (e.g., colorization, image inpainting) within a 2D image editor of their choice, and in the next step, our technique produces a photorealistic result that approximates this target image. Our method combines the advantages of a neural network-augmented optimizer and an encoder neural network to produce high-quality output results within 30 seconds. We also demonstrate that it is resilient against poorly-edited target images and propose a simple extension to predict image sequences with a strict time budget of 1-2 seconds per image.