Unsupervised Structural Scene Decomposition via Foreground-Aware Slot Attention with Pseudo-Mask Guidance

TL;DR

This paper introduces FASA, a novel unsupervised scene decomposition method that explicitly separates foreground and background using a two-stage process with pseudo-mask guidance, improving object discovery in complex scenes.

Contribution

FASA is the first to explicitly model foreground-background separation in unsupervised scene decomposition with a dual-stage approach and pseudo-mask guidance, enhancing object representation accuracy.

Findings

FASA outperforms existing methods on synthetic and real datasets.

Explicit foreground modeling improves scene decomposition quality.

Pseudo-mask guidance reduces over-segmentation of objects.

Abstract

Recent advances in object-centric representation learning have shown that slot attention-based methods can effectively decompose visual scenes into object slot representations without supervision. However, existing approaches typically process foreground and background regions indiscriminately, often resulting in background interference and suboptimal instance discovery performance on real-world data. To address this limitation, we propose Foreground-Aware Slot Attention (FASA), a two-stage framework that explicitly separates foreground from background to enable precise object discovery. In the first stage, FASA performs a coarse scene decomposition to distinguish foreground from background regions through a dual-slot competition mechanism. These slots are initialized via a clustering-based strategy, yielding well-structured representations of salient regions. In the second stage, we introduce a masked slot attention mechanism where the first slot captures the background while the remaining slots compete to represent individual foreground objects. To further address over-segmentation of foreground objects, we incorporate pseudo-mask guidance derived from a patch affinity graph constructed with self-supervised image features to guide the learning of foreground slots. Extensive experiments on both synthetic and real-world datasets demonstrate that FASA consistently outperforms state-of-the-art methods, validating the effectiveness of explicit foreground modeling and pseudo-mask guidance for robust scene decomposition and object-coherent representation. Code will be made publicly available.