StyMam: A Mamba-Based Generator for Artistic Style Transfer

TL;DR

StyMam introduces a novel mamba-based generator with dual-path and attention mechanisms for high-quality, artifact-free artistic style transfer, outperforming existing GAN and diffusion methods in quality and speed.

Contribution

The paper proposes a mamba-based generator with residual dual-path scanning and spatial attention for improved style transfer, addressing limitations of GAN and diffusion approaches.

Findings

Outperforms state-of-the-art methods in quality and speed

Produces artifact-free stylized images

Effectively captures local textures and global dependencies

Abstract

Image style transfer aims to integrate the visual patterns of a specific artistic style into a content image while preserving its content structure. Existing methods mainly rely on the generative adversarial network (GAN) or stable diffusion (SD). GAN-based approaches using CNNs or Transformers struggle to jointly capture local and global dependencies, leading to artifacts and disharmonious patterns. SD-based methods reduce such issues but often fail to preserve content structures and suffer from slow inference. To address these issues, we revisit GAN and propose a mamba-based generator, termed as StyMam, to produce high-quality stylized images without introducing artifacts and disharmonious patterns. Specifically, we introduce a mamba-based generator with a residual dual-path strip scanning mechanism and a channel-reweighted spatial attention module. The former efficiently captures local texture features, while the latter models global dependencies. Finally, extensive qualitative and quantitative experiments demonstrate that the proposed method outperforms state-of-the-art algorithms in both quality and speed.