SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fields

TL;DR

SlotLifter introduces a novel object-centric radiance model that combines scene reconstruction and decomposition with slot-guided feature lifting, achieving state-of-the-art results in 3D scene understanding and rendering.

Contribution

It unites object-centric learning with image-based rendering, providing a new approach that significantly improves scene decomposition and view synthesis performance.

Findings

Outperforms existing 3D object-centric methods by a large margin

Achieves state-of-the-art results on synthetic and real-world datasets

Demonstrates effectiveness through extensive ablative studies

Abstract

The ability to distill object-centric abstractions from intricate visual scenes underpins human-level generalization. Despite the significant progress in object-centric learning methods, learning object-centric representations in the 3D physical world remains a crucial challenge. In this work, we propose SlotLifter, a novel object-centric radiance model addressing scene reconstruction and decomposition jointly via slot-guided feature lifting. Such a design unites object-centric learning representations and image-based rendering methods, offering state-of-the-art performance in scene decomposition and novel-view synthesis on four challenging synthetic and four complex real-world datasets, outperforming existing 3D object-centric learning methods by a large margin. Through extensive ablative studies, we showcase the efficacy of designs in SlotLifter, revealing key insights for potential future directions.