Frequency-Domain Fusion Transformer for Image Inpainting

TL;DR

This paper introduces a frequency-domain fusion transformer that enhances image inpainting by better preserving high-frequency details and textures through wavelet, Gabor filtering, and Fourier-based adaptive filtering.

Contribution

It proposes a novel Transformer-based inpainting method integrating frequency-domain techniques for improved detail preservation and texture modeling.

Findings

Improved high-frequency detail preservation in inpainted images

Enhanced texture and structural modeling through frequency-domain fusion

Outperforms existing methods in visual quality and accuracy

Abstract

Image inpainting plays a vital role in restoring missing image regions and supporting high-level vision tasks, but traditional methods struggle with complex textures and large occlusions. Although Transformer-based approaches have demonstrated strong global modeling capabilities, they often fail to preserve high-frequency details due to the low-pass nature of self-attention and suffer from high computational costs. To address these challenges, this paper proposes a Transformer-based image inpainting method incorporating frequency-domain fusion. Specifically, an attention mechanism combining wavelet transform and Gabor filtering is introduced to enhance multi-scale structural modeling and detail preservation. Additionally, a learnable frequency-domain filter based on the fast Fourier transform is designed to replace the feedforward network, enabling adaptive noise suppression and detail retention. The model adopts a four-level encoder-decoder structure and is guided by a novel loss strategy to balance global semantics and fine details. Experimental results demonstrate that the proposed method effectively improves the quality of image inpainting by preserving more high-frequency information.