FIRM: Flexible Interactive Reflection reMoval

TL;DR

FIRM introduces a flexible, interactive reflection removal framework that accepts various guidance forms, significantly reducing user effort while achieving state-of-the-art results in separating reflection layers from images.

Contribution

The paper proposes a novel unified guidance conversion module and a contrastive mask-guided network with a cross-attention mechanism, enabling flexible, efficient, and accurate reflection removal with minimal user input.

Findings

Requires only 10% of the guidance time compared to previous methods

Achieves state-of-the-art reflection removal performance on real-world datasets

Supports multiple guidance modalities such as points, boxes, strokes, and text

Abstract

Removing reflection from a single image is challenging due to the absence of general reflection priors. Although existing methods incorporate extensive user guidance for satisfactory performance, they often lack the flexibility to adapt user guidance in different modalities, and dense user interactions further limit their practicality. To alleviate these problems, this paper presents FIRM, a novel framework for Flexible Interactive image Reflection reMoval with various forms of guidance, where users can provide sparse visual guidance (e.g., points, boxes, or strokes) or text descriptions for better reflection removal. Firstly, we design a novel user guidance conversion module (UGC) to transform different forms of guidance into unified contrastive masks. The contrastive masks provide explicit cues for identifying reflection and transmission layers in blended images. Secondly, we devise a contrastive mask-guided reflection removal network that comprises a newly proposed contrastive guidance interaction block (CGIB). This block leverages a unique cross-attention mechanism that merges contrastive masks with image features, allowing for precise layer separation. The proposed framework requires only 10\% of the guidance time needed by previous interactive methods, which makes a step-change in flexibility. Extensive results on public real-world reflection removal datasets validate that our method demonstrates state-of-the-art reflection removal performance. Code is avaliable at https://github.com/ShawnChenn/FlexibleReflectionRemoval.