Tool- and Domain-Agnostic Parameterization of Style Transfer Effects Leveraging Pretrained Perceptual Metrics

TL;DR

This paper introduces a method to convert style transfer effects into adjustable parameters within existing design tools, enabling more flexible and exploratory style editing by leveraging pretrained perceptual metrics.

Contribution

We propose a novel parametric transcription framework that translates deep learning style transfer effects into tool-specific parameters using perceptual metrics and black-box optimization.

Findings

Effective style imitation in third-party tools like Instagram and Blender.

Enables user-driven exploration by adjusting style parameters.

Leverages pretrained perceptual models for style distance measurement.

Abstract

Current deep learning techniques for style transfer would not be optimal for design support since their "one-shot" transfer does not fit exploratory design processes. To overcome this gap, we propose parametric transcription, which transcribes an end-to-end style transfer effect into parameter values of specific transformations available in an existing content editing tool. With this approach, users can imitate the style of a reference sample in the tool that they are familiar with and thus can easily continue further exploration by manipulating the parameters. To enable this, we introduce a framework that utilizes an existing pretrained model for style transfer to calculate a perceptual style distance to the reference sample and uses black-box optimization to find the parameters that minimize this distance. Our experiments with various third-party tools, such as Instagram and Blender, show that our framework can effectively leverage deep learning techniques for computational design support.