HSI: A Holistic Style Injector for Arbitrary Style Transfer

TL;DR

This paper introduces HSI, a novel style transfer module that emphasizes global style features, reduces computational complexity, and improves stylization quality by leveraging semantic relations.

Contribution

The paper proposes a holistic style injector that focuses on global style representation, employs a dual relation learning mechanism, and achieves linear complexity in style transfer.

Findings

Outperforms state-of-the-art methods in effectiveness.

Reduces computational complexity to linear scale.

Enhances style fidelity and content preservation.

Abstract

Attention-based arbitrary style transfer methods have gained significant attention recently due to their impressive ability to synthesize style details. However, the point-wise matching within the attention mechanism may overly focus on local patterns such that neglect the remarkable global features of style images. Additionally, when processing large images, the quadratic complexity of the attention mechanism will bring high computational load. To alleviate above problems, we propose Holistic Style Injector (HSI), a novel attention-style transformation module to deliver artistic expression of target style. Specifically, HSI performs stylization only based on global style representation that is more in line with the characteristics of style transfer, to avoid generating local disharmonious patterns in stylized images. Moreover, we propose a dual relation learning mechanism inside the HSI to dynamically render images by leveraging semantic similarity in content and style, ensuring the stylized images preserve the original content and improve style fidelity. Note that the proposed HSI achieves linear computational complexity because it establishes feature mapping through element-wise multiplication rather than matrix multiplication. Qualitative and quantitative results demonstrate that our method outperforms state-of-the-art approaches in both effectiveness and efficiency.