PCA-Based Knowledge Distillation Towards Lightweight and Content-Style Balanced Photorealistic Style Transfer Models

TL;DR

This paper introduces a novel PCA-based knowledge distillation method to create lightweight, fast, and content-style balanced photorealistic style transfer models, significantly improving speed and efficiency.

Contribution

It is the first to apply knowledge distillation to photorealistic style transfer, enabling smaller models with better performance across multiple architectures and resolutions.

Findings

Models run 5-20x faster than existing methods.

Distilled models use at most 1% of parameters of original models.

Achieves better stylization and content preservation balance.

Abstract

Photorealistic style transfer entails transferring the style of a reference image to another image so the result seems like a plausible photo. Our work is inspired by the observation that existing models are slow due to their large sizes. We introduce PCA-based knowledge distillation to distill lightweight models and show it is motivated by theory. To our knowledge, this is the first knowledge distillation method for photorealistic style transfer. Our experiments demonstrate its versatility for use with different backbone architectures, VGG and MobileNet, across six image resolutions. Compared to existing models, our top-performing model runs at speeds 5-20x faster using at most 1\% of the parameters. Additionally, our distilled models achieve a better balance between stylization strength and content preservation than existing models. To support reproducing our method and models, we share the code at \textit{https://github.com/chiutaiyin/PCA-Knowledge-Distillation}.