Understanding image motion with group representations

TL;DR

This paper introduces a group-theoretic approach to learning image motion representations from unlabeled video, enabling the extraction of useful motion features for tasks like localization and odometry without labeled data.

Contribution

It proposes a novel method that constrains neural networks with group properties of transformations to learn motion representations from unlabeled videos.

Findings

Captures motion in synthetic and real-world sequences without labels.

Extracts useful information for localization, tracking, and odometry.

Networks respect group constraints to implicitly identify motion characteristics.

Abstract

Motion is an important signal for agents in dynamic environments, but learning to represent motion from unlabeled video is a difficult and underconstrained problem. We propose a model of motion based on elementary group properties of transformations and use it to train a representation of image motion. While most methods of estimating motion are based on pixel-level constraints, we use these group properties to constrain the abstract representation of motion itself. We demonstrate that a deep neural network trained using this method captures motion in both synthetic 2D sequences and real-world sequences of vehicle motion, without requiring any labels. Networks trained to respect these constraints implicitly identify the image characteristic of motion in different sequence types. In the context of vehicle motion, this method extracts information useful for localization, tracking, and odometry. Our results demonstrate that this representation is useful for learning motion in the general setting where explicit labels are difficult to obtain.