StyleMamba : State Space Model for Efficient Text-driven Image Style Transfer

TL;DR

StyleMamba is an efficient text-guided image style transfer framework that reduces training and inference time by using a state space model and novel loss functions, achieving superior stylization quality.

Contribution

It introduces a conditional State Space Model for faster, resource-efficient text-driven style transfer with enhanced style consistency.

Findings

Reduces training iterations by 5 times

Cuts inference time by 3 times

Achieves superior stylization performance

Abstract

We present StyleMamba, an efficient image style transfer framework that translates text prompts into corresponding visual styles while preserving the content integrity of the original images. Existing text-guided stylization requires hundreds of training iterations and takes a lot of computing resources. To speed up the process, we propose a conditional State Space Model for Efficient Text-driven Image Style Transfer, dubbed StyleMamba, that sequentially aligns the image features to the target text prompts. To enhance the local and global style consistency between text and image, we propose masked and second-order directional losses to optimize the stylization direction to significantly reduce the training iterations by 5 times and the inference time by 3 times. Extensive experiments and qualitative evaluation confirm the robust and superior stylization performance of our methods compared to the existing baselines.