MUFASA: A Multi-Layer Framework for Slot Attention

TL;DR

MUFASA enhances unsupervised object segmentation by leveraging multi-layer semantic information from vision transformers, improving accuracy and convergence with minimal overhead.

Contribution

Introduces MUFASA, a multi-layer framework that integrates semantic-rich features from all ViT layers into slot attention for better object segmentation.

Findings

Achieves state-of-the-art segmentation results on multiple datasets.

Improves training convergence speed.

Adds minimal inference overhead.

Abstract

Unsupervised object-centric learning (OCL) decomposes visual scenes into distinct entities. Slot attention is a popular approach that represents individual objects as latent vectors, called slots. Current methods obtain these slot representations solely from the last layer of a pre-trained vision transformer (ViT), ignoring valuable, semantically rich information encoded across the other layers. To better utilize this latent semantic information, we introduce MUFASA, a lightweight plug-and-play framework for slot attention-based approaches to unsupervised object segmentation. Our model computes slot attention across multiple feature layers of the ViT encoder, fully leveraging their semantic richness. We propose a fusion strategy to aggregate slots obtained on multiple layers into a unified object-centric representation. Integrating MUFASA into existing OCL methods improves their segmentation results across multiple datasets, setting a new state of the art while simultaneously improving training convergence with only minor inference overhead.