SCOD: Active Object Detection for Embodied Agents using Sensory Commutativity of Action Sequences

TL;DR

SCOD is an active object detection method that leverages the commutative properties of action sequences in embodied agents, demonstrating high accuracy and generalization in both simulated and real environments.

Contribution

We propose SCOD, a novel active detection approach using action sequence commutativity, applicable to both movable and immovable objects in embodied agent scenarios.

Findings

High accuracy in 3D realistic robotic setups

Effective generalization to unseen environments and objects

Successful real robot application demonstrating practicality

Abstract

We introduce SCOD (Sensory Commutativity Object Detection), an active method for movable and immovable object detection. SCOD exploits the commutative properties of action sequences, in the scenario of an embodied agent equipped with first-person sensors and a continuous motor space with multiple degrees of freedom. SCOD is based on playing an action sequence in two different orders from the same starting point and comparing the two final observations obtained after each sequence. Our experiments on 3D realistic robotic setups (iGibson) demonstrate the accuracy of SCOD and its generalization to unseen environments and objects. We also successfully apply SCOD on a real robot to further illustrate its generalization properties. With SCOD, we aim at providing a novel way of approaching the problem of object discovery in the context of a naive embodied agent. We provide code and a supplementary video.