Top-Down Guidance for Learning Object-Centric Representations

TL;DR

This paper introduces TDGNet, a top-down guided network that enhances object-centric representations by incorporating high-level guidance during training and inference, leading to improved performance on complex datasets and downstream robotics tasks.

Contribution

The paper proposes a novel top-down guidance mechanism for object-centric learning, addressing limitations of existing models by improving object distinction and expanding downstream application scope.

Findings

TDGNet outperforms existing models on multiple datasets.

It enhances object distinction in complex scenes.

Effective in robotics tasks like video prediction and visual planning.

Abstract

Humans' innate ability to decompose scenes into objects allows for efficient understanding, predicting, and planning. In light of this, Object-Centric Learning (OCL) attempts to endow networks with similar capabilities, learning to represent scenes with the composition of objects. However, existing OCL models only learn through reconstructing the input images, which does not assist the model in distinguishing objects, resulting in suboptimal object-centric representations. This flaw limits current object-centric models to relatively simple downstream tasks. To address this issue, we draw on humans' top-down vision pathway and propose Top-Down Guided Network (TDGNet), which includes a top-down pathway to improve object-centric representations. During training, the top-down pathway constructs guidance with high-level object-centric representations to optimize low-level grid features output by the backbone. While during inference, it refines object-centric representations by detecting and solving conflicts between low- and high-level features. We show that TDGNet outperforms current object-centric models on multiple datasets of varying complexity. In addition, we expand the downstream task scope of object-centric representations by applying TDGNet to the field of robotics, validating its effectiveness in downstream tasks including video prediction and visual planning.