F2T2-HiT: A U-Shaped FFT Transformer and Hierarchical Transformer for Reflection Removal

TL;DR

This paper introduces F2T2-HiT, a novel Transformer-based architecture combining FFT and Hierarchical Transformers within a U-Net framework to effectively remove reflections from single images, achieving state-of-the-art results.

Contribution

The paper proposes a new U-shaped FFT and Hierarchical Transformer architecture specifically designed for single image reflection removal, integrating frequency domain and multi-scale features.

Findings

Achieves state-of-the-art performance on three datasets.

Effectively captures reflection patterns using FFT Transformer blocks.

Handles varying reflection sizes with Hierarchical Transformer blocks.

Abstract

Single Image Reflection Removal (SIRR) technique plays a crucial role in image processing by eliminating unwanted reflections from the background. These reflections, often caused by photographs taken through glass surfaces, can significantly degrade image quality. SIRR remains a challenging problem due to the complex and varied reflections encountered in real-world scenarios. These reflections vary significantly in intensity, shapes, light sources, sizes, and coverage areas across the image, posing challenges for most existing methods to effectively handle all cases. To address these challenges, this paper introduces a U-shaped Fast Fourier Transform Transformer and Hierarchical Transformer (F2T2-HiT) architecture, an innovative Transformer-based design for SIRR. Our approach uniquely combines Fast Fourier Transform (FFT) Transformer blocks and Hierarchical Transformer blocks within a UNet framework. The FFT Transformer blocks leverage the global frequency domain information to effectively capture and separate reflection patterns, while the Hierarchical Transformer blocks utilize multi-scale feature extraction to handle reflections of varying sizes and complexities. Extensive experiments conducted on three publicly available testing datasets demonstrate state-of-the-art performance, validating the effectiveness of our approach.