Is an object-centric representation beneficial for robotic manipulation ?

TL;DR

This paper investigates whether object-centric representations improve robotic manipulation by evaluating their generalization capabilities in complex, multi-object simulated environments, comparing them to holistic approaches.

Contribution

The study provides a systematic evaluation of object-centric representations in robotic manipulation tasks, highlighting their advantages over holistic methods in complex scenarios.

Findings

Object-centric methods outperform holistic approaches in complex scenes.

Existing object-centric models struggle with scene complexity and inter-object interactions.

Object-centric representations enhance generalization in multi-object robotic tasks.

Abstract

Object-centric representation (OCR) has recently become a subject of interest in the computer vision community for learning a structured representation of images and videos. It has been several times presented as a potential way to improve data-efficiency and generalization capabilities to learn an agent on downstream tasks. However, most existing work only evaluates such models on scene decomposition, without any notion of reasoning over the learned representation. Robotic manipulation tasks generally involve multi-object environments with potential inter-object interaction. We thus argue that they are a very interesting playground to really evaluate the potential of existing object-centric work. To do so, we create several robotic manipulation tasks in simulated environments involving multiple objects (several distractors, the robot, etc.) and a high-level of randomization (object positions, colors, shapes, background, initial positions, etc.). We then evaluate one classical object-centric method across several generalization scenarios and compare its results against several state-of-the-art hollistic representations. Our results exhibit that existing methods are prone to failure in difficult scenarios involving complex scene structures, whereas object-centric methods help overcome these challenges.