Making a Case for Learning Motion Representations with Phase

TL;DR

This paper advocates for Eulerian motion representation learning using phase, derived from complex-steerable pyramids, demonstrating its advantages across action recognition, motion prediction, and transfer tasks in images and videos.

Contribution

It introduces phase-based Eulerian motion representations as an alternative to optical flow, with practical applications in various motion-related tasks.

Findings

Phase-based Eulerian motion captures motion effectively.

Improves action recognition accuracy.

Enables motion transfer in static images and videos.

Abstract

This work advocates Eulerian motion representation learning over the current standard Lagrangian optical flow model. Eulerian motion is well captured by using phase, as obtained by decomposing the image through a complex-steerable pyramid. We discuss the gain of Eulerian motion in a set of practical use cases: (i) action recognition, (ii) motion prediction in static images, (iii) motion transfer in static images and, (iv) motion transfer in video. For each task we motivate the phase-based direction and provide a possible approach.