FastCLIPstyler: Optimisation-free Text-based Image Style Transfer Using Style Representations

TL;DR

FastCLIPstyler introduces a rapid, optimization-free text-based image style transfer method that produces high-quality stylisations in a single forward pass, suitable for resource-constrained devices.

Contribution

It presents FastCLIPstyler, a novel model enabling instant style transfer from text descriptions, and EdgeCLIPstyler, a lightweight version for edge devices, improving speed and accessibility.

Findings

Achieves superior stylisation quality compared to state-of-the-art methods.

Significantly reduces runtime, enabling real-time applications.

Demonstrates effectiveness on resource-constrained devices.

Abstract

In recent years, language-driven artistic style transfer has emerged as a new type of style transfer technique, eliminating the need for a reference style image by using natural language descriptions of the style. The first model to achieve this, called CLIPstyler, has demonstrated impressive stylisation results. However, its lengthy optimisation procedure at runtime for each query limits its suitability for many practical applications. In this work, we present FastCLIPstyler, a generalised text-based image style transfer model capable of stylising images in a single forward pass for arbitrary text inputs. Furthermore, we introduce EdgeCLIPstyler, a lightweight model designed for compatibility with resource-constrained devices. Through quantitative and qualitative comparisons with state-of-the-art approaches, we demonstrate that our models achieve superior stylisation quality based on measurable metrics while offering significantly improved runtime efficiency, particularly on edge devices.