Multi-Object Discovery by Low-Dimensional Object Motion

TL;DR

This paper introduces a method that models pixel-wise geometry and object motion to improve unsupervised multi-object segmentation and monocular depth estimation by constraining motion prediction to a low-dimensional scene structure.

Contribution

It proposes a novel approach that uses depth and scene structure to reduce ambiguity in motion prediction from a single image, achieving state-of-the-art results.

Findings

State-of-the-art unsupervised multi-object segmentation performance.

Reliable monocular depth estimation results.

Effective modeling of scene structure and object motion.

Abstract

Recent work in unsupervised multi-object segmentation shows impressive results by predicting motion from a single image despite the inherent ambiguity in predicting motion without the next image. On the other hand, the set of possible motions for an image can be constrained to a low-dimensional space by considering the scene structure and moving objects in it. We propose to model pixel-wise geometry and object motion to remove ambiguity in reconstructing flow from a single image. Specifically, we divide the image into coherently moving regions and use depth to construct flow bases that best explain the observed flow in each region. We achieve state-of-the-art results in unsupervised multi-object segmentation on synthetic and real-world datasets by modeling the scene structure and object motion. Our evaluation of the predicted depth maps shows reliable performance in monocular depth estimation.