LFSamba: Marry SAM with Mamba for Light Field Salient Object Detection

TL;DR

LFSamba introduces a novel light field salient object detection model that combines SAM and Mamba for efficient feature extraction, multi-modal and inter-slice relation modeling, and weakly supervised learning, advancing the state-of-the-art in light field analysis.

Contribution

The paper presents a new model LFSamba that integrates SAM and Mamba to improve multi-focus light field salient object detection with weak supervision.

Findings

Effective feature extraction with SAM

Long-range inter-slice dependency modeling with Mamba

First scribble-supervised baseline for light field detection

Abstract

A light field camera can reconstruct 3D scenes using captured multi-focus images that contain rich spatial geometric information, enhancing applications in stereoscopic photography, virtual reality, and robotic vision. In this work, a state-of-the-art salient object detection model for multi-focus light field images, called LFSamba, is introduced to emphasize four main insights: (a) Efficient feature extraction, where SAM is used to extract modality-aware discriminative features; (b) Inter-slice relation modeling, leveraging Mamba to capture long-range dependencies across multiple focal slices, thus extracting implicit depth cues; (c) Inter-modal relation modeling, utilizing Mamba to integrate all-focus and multi-focus images, enabling mutual enhancement; (d) Weakly supervised learning capability, developing a scribble annotation dataset from an existing pixel-level mask dataset, establishing the first scribble-supervised baseline for light field salient object detection.https://github.com/liuzywen/LFScribble