Dimensions of Motion: Monocular Prediction through Flow Subspaces

TL;DR

This paper presents a novel unsupervised method for scene representation learning from monocular videos by predicting flow subspaces, enabling tasks like depth prediction without requiring explicit camera information.

Contribution

It introduces a flow subspace prediction approach with a new loss function, allowing unsupervised learning of scene representations from in-the-wild videos.

Findings

Achieves comparable indoor depth prediction performance to supervised methods

Enables unsupervised scene understanding without camera pose or stereo data

Provides a flexible framework for monocular scene representation learning

Abstract

We introduce a way to learn to estimate a scene representation from a single image by predicting a low-dimensional subspace of optical flow for each training example, which encompasses the variety of possible camera and object movement. Supervision is provided by a novel loss which measures the distance between this predicted flow subspace and an observed optical flow. This provides a new approach to learning scene representation tasks, such as monocular depth prediction or instance segmentation, in an unsupervised fashion using in-the-wild input videos without requiring camera poses, intrinsics, or an explicit multi-view stereo step. We evaluate our method in multiple settings, including an indoor depth prediction task where it achieves comparable performance to recent methods trained with more supervision.