CFFT-GAN: Cross-domain Feature Fusion Transformer for Exemplar-based Image Translation

TL;DR

CFFT-GAN introduces a novel transformer-based approach for exemplar-based image translation that effectively fuses inter- and intra-domain features, outperforming existing methods in quality and flexibility.

Contribution

The paper proposes the Cross-domain Feature Fusion Transformer (CFFT) and integrates it into a GAN framework, enabling comprehensive feature interaction modeling for improved image translation.

Findings

CFFT-GAN achieves superior results on multiple image translation tasks.

The ablation studies confirm the effectiveness of the CFFT module.

The method demonstrates strong potential for multi-domain feature fusion.

Abstract

Exemplar-based image translation refers to the task of generating images with the desired style, while conditioning on certain input image. Most of the current methods learn the correspondence between two input domains and lack the mining of information within the domains. In this paper, we propose a more general learning approach by considering two domain features as a whole and learning both inter-domain correspondence and intra-domain potential information interactions. Specifically, we propose a Cross-domain Feature Fusion Transformer (CFFT) to learn inter- and intra-domain feature fusion. Based on CFFT, the proposed CFFT-GAN works well on exemplar-based image translation. Moreover, CFFT-GAN is able to decouple and fuse features from multiple domains by cascading CFFT modules. We conduct rich quantitative and qualitative experiments on several image translation tasks, and the results demonstrate the superiority of our approach compared to state-of-the-art methods. Ablation studies show the importance of our proposed CFFT. Application experimental results reflect the potential of our method.