Object Concepts Emerge from Motion

TL;DR

This paper introduces a biologically inspired, unsupervised framework that leverages motion boundaries in videos to learn object-centric visual representations, outperforming existing methods across multiple vision tasks.

Contribution

It proposes a novel, label-free approach using motion cues for object representation learning, scalable to large unstructured video data.

Findings

Outperforms previous supervised and self-supervised methods

Demonstrates strong generalization to unseen scenes

Effective across both low-level and high-level vision tasks

Abstract

Object concepts play a foundational role in human visual cognition, enabling perception, memory, and interaction in the physical world. Inspired by findings in developmental neuroscience - where infants are shown to acquire object understanding through observation of motion - we propose a biologically inspired framework for learning object-centric visual representations in an unsupervised manner. Our key insight is that motion boundary serves as a strong signal for object-level grouping, which can be used to derive pseudo instance supervision from raw videos. Concretely, we generate motion-based instance masks using off-the-shelf optical flow and clustering algorithms, and use them to train visual encoders via contrastive learning. Our framework is fully label-free and does not rely on camera calibration, making it scalable to large-scale unstructured video data. We evaluate our approach on three downstream tasks spanning both low-level (monocular depth estimation) and high-level (3D object detection and occupancy prediction) vision. Our models outperform previous supervised and self-supervised baselines and demonstrate strong generalization to unseen scenes. These results suggest that motion-induced object representations offer a compelling alternative to existing vision foundation models, capturing a crucial but overlooked level of abstraction: the visual instance. The corresponding code will be released upon paper acceptance.