SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models

TL;DR

SlotDiffusion introduces an object-centric latent diffusion model that significantly improves unsupervised object segmentation and high-quality image and video generation, surpassing previous methods and demonstrating scalability to real-world datasets.

Contribution

It presents SlotDiffusion, a novel diffusion-based approach for object-centric generative modeling that enhances image quality and segmentation accuracy over prior slot-based models.

Findings

Outperforms previous slot models in segmentation and generation across six datasets.

Improves video prediction quality and temporal reasoning tasks.

Scales effectively to real-world datasets like PASCAL VOC and COCO.

Abstract

Object-centric learning aims to represent visual data with a set of object entities (a.k.a. slots), providing structured representations that enable systematic generalization. Leveraging advanced architectures like Transformers, recent approaches have made significant progress in unsupervised object discovery. In addition, slot-based representations hold great potential for generative modeling, such as controllable image generation and object manipulation in image editing. However, current slot-based methods often produce blurry images and distorted objects, exhibiting poor generative modeling capabilities. In this paper, we focus on improving slot-to-image decoding, a crucial aspect for high-quality visual generation. We introduce SlotDiffusion -- an object-centric Latent Diffusion Model (LDM) designed for both image and video data. Thanks to the powerful modeling capacity of LDMs, SlotDiffusion surpasses previous slot models in unsupervised object segmentation and visual generation across six datasets. Furthermore, our learned object features can be utilized by existing object-centric dynamics models, improving video prediction quality and downstream temporal reasoning tasks. Finally, we demonstrate the scalability of SlotDiffusion to unconstrained real-world datasets such as PASCAL VOC and COCO, when integrated with self-supervised pre-trained image encoders.