Retinex-guided Channel-grouping based Patch Swap for Arbitrary Style Transfer

TL;DR

This paper introduces a novel style transfer method using Retinex-guided channel grouping and patch swapping, which improves style consistency and content preservation over existing techniques.

Contribution

It proposes a Retinex-based channel grouping and patch swap approach that better captures style diversity and enhances stylization quality.

Findings

Outperforms existing methods in style consistency.

Maintains higher content fidelity.

Reduces artifacts like black areas and over-stylization.

Abstract

The basic principle of the patch-matching based style transfer is to substitute the patches of the content image feature maps by the closest patches from the style image feature maps. Since the finite features harvested from one single aesthetic style image are inadequate to represent the rich textures of the content natural image, existing techniques treat the full-channel style feature patches as simple signal tensors and create new style feature patches via signal-level fusion, which ignore the implicit diversities existed in style features and thus fail for generating better stylised results. In this paper, we propose a Retinex theory guided, channel-grouping based patch swap technique to solve the above challenges. Channel-grouping strategy groups the style feature maps into surface and texture channels, which prevents the winner-takes-all problem. Retinex theory based decomposition controls a more stable channel code rate generation. In addition, we provide complementary fusion and multi-scale generation strategy to prevent unexpected black area and over-stylised results respectively. Experimental results demonstrate that the proposed method outperforms the existing techniques in providing more style-consistent textures while keeping the content fidelity.