DCT-Net: Domain-Calibrated Translation for Portrait Stylization

TL;DR

DCT-Net is a new image translation architecture designed for few-shot portrait stylization, capable of producing high-quality, generalizable stylized images even with limited style exemplars, by calibrating content and leveraging local-global structure.

Contribution

The paper introduces DCT-Net, a novel architecture that improves few-shot portrait stylization through content calibration, geometric expansion, and texture translation modules, enabling high-fidelity and versatile stylization.

Findings

Outperforms state-of-the-art head stylization methods.

Effective in full-body image translation with adaptive deformations.

Handles complex scenes with occlusions and accessories.

Abstract

This paper introduces DCT-Net, a novel image translation architecture for few-shot portrait stylization. Given limited style exemplars ($\sim$100), the new architecture can produce high-quality style transfer results with advanced ability to synthesize high-fidelity contents and strong generality to handle complicated scenes (e.g., occlusions and accessories). Moreover, it enables full-body image translation via one elegant evaluation network trained by partial observations (i.e., stylized heads). Few-shot learning based style transfer is challenging since the learned model can easily become overfitted in the target domain, due to the biased distribution formed by only a few training examples. This paper aims to handle the challenge by adopting the key idea of "calibration first, translation later" and exploring the augmented global structure with locally-focused translation. Specifically, the proposed DCT-Net consists of three modules: a content adapter borrowing the powerful prior from source photos to calibrate the content distribution of target samples; a geometry expansion module using affine transformations to release spatially semantic constraints; and a texture translation module leveraging samples produced by the calibrated distribution to learn a fine-grained conversion. Experimental results demonstrate the proposed method's superiority over the state of the art in head stylization and its effectiveness on full image translation with adaptive deformations.