MirrorMamba: Towards Scalable and Robust Mirror Detection in Videos

TL;DR

MirrorMamba introduces a scalable, robust video mirror detection method leveraging multiple cues and a novel Mamba-based architecture, outperforming existing approaches in accuracy and generalizability across datasets.

Contribution

The paper presents the first application of Mamba architecture in mirror detection, combining multiple cues and innovative modules for improved performance and robustness.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Achieves top results on challenging image-based mirror detection datasets.

Demonstrates robustness and generalizability across diverse conditions.

Abstract

Video mirror detection has received significant research attention, yet existing methods suffer from limited performance and robustness. These approaches often over-rely on single, unreliable dynamic features, and are typically built on CNNs with limited receptive fields or Transformers with quadratic computational complexity. To address these limitations, we propose a new effective and scalable video mirror detection method, called MirrorMamba. Our approach leverages multiple cues to adapt to diverse conditions, incorporating perceived depth, correspondence and optical. We also introduce an innovative Mamba-based Multidirection Correspondence Extractor, which benefits from the global receptive field and linear complexity of the emerging Mamba spatial state model to effectively capture correspondence properties. Additionally, we design a Mamba-based layer-wise boundary enforcement decoder to resolve the unclear boundary caused by the blurred depth map. Notably, this work marks the first successful application of the Mamba-based architecture in the field of mirror detection. Extensive experiments demonstrate that our method outperforms existing state-of-the-art approaches for video mirror detection on the benchmark datasets. Furthermore, on the most challenging and representative image-based mirror detection dataset, our approach achieves state-of-the-art performance, proving its robustness and generalizability.