WISE: Whitebox Image Stylization by Example-based Learning

TL;DR

WISE introduces an example-based, differentiable image stylization system that learns to adapt traditional heuristic filters to specific styles, enabling high-resolution artistic rendering with interpretability and flexibility.

Contribution

The paper presents WISE, a novel framework that makes heuristic image filters differentiable and trainable for style adaptation, bridging classical filtering and deep learning techniques.

Findings

Achieves style transfer comparable to state-of-the-art GANs.

Supports multiple stylization techniques within a unified framework.

Enhances high-resolution image stylization with interpretable parameters.

Abstract

Image-based artistic rendering can synthesize a variety of expressive styles using algorithmic image filtering. In contrast to deep learning-based methods, these heuristics-based filtering techniques can operate on high-resolution images, are interpretable, and can be parameterized according to various design aspects. However, adapting or extending these techniques to produce new styles is often a tedious and error-prone task that requires expert knowledge. We propose a new paradigm to alleviate this problem: implementing algorithmic image filtering techniques as differentiable operations that can learn parametrizations aligned to certain reference styles. To this end, we present WISE, an example-based image-processing system that can handle a multitude of stylization techniques, such as watercolor, oil or cartoon stylization, within a common framework. By training parameter prediction networks for global and local filter parameterizations, we can simultaneously adapt effects to reference styles and image content, e.g., to enhance facial features. Our method can be optimized in a style-transfer framework or learned in a generative-adversarial setting for image-to-image translation. We demonstrate that jointly training an XDoG filter and a CNN for postprocessing can achieve comparable results to a state-of-the-art GAN-based method.