ProMotion: Prototypes As Motion Learners

TL;DR

ProMotion introduces a unified prototypical framework for modeling diverse motion tasks, improving robustness and transferability across 2D and 3D applications, outperforming specialized architectures.

Contribution

It presents a novel unified paradigm using prototypes and a dual mechanism to model motion, enhancing robustness and transferability across various tasks.

Findings

Achieves state-of-the-art results on Sintel and KITTI datasets.

Demonstrates robustness and transferability across multiple motion tasks.

Outperforms existing specialized architectures in depth and flow estimation.

Abstract

In this work, we introduce ProMotion, a unified prototypical framework engineered to model fundamental motion tasks. ProMotion offers a range of compelling attributes that set it apart from current task-specific paradigms. We adopt a prototypical perspective, establishing a unified paradigm that harmonizes disparate motion learning approaches. This novel paradigm streamlines the architectural design, enabling the simultaneous assimilation of diverse motion information. We capitalize on a dual mechanism involving the feature denoiser and the prototypical learner to decipher the intricacies of motion. This approach effectively circumvents the pitfalls of ambiguity in pixel-wise feature matching, significantly bolstering the robustness of motion representation. We demonstrate a profound degree of transferability across distinct motion patterns. This inherent versatility reverberates robustly across a comprehensive spectrum of both 2D and 3D downstream tasks. Empirical results demonstrate that ProMotion outperforms various well-known specialized architectures, achieving 0.54 and 0.054 Abs Rel error on the Sintel and KITTI depth datasets, 1.04 and 2.01 average endpoint error on the clean and final pass of Sintel flow benchmark, and 4.30 F1-all error on the KITTI flow benchmark. For its efficacy, we hope our work can catalyze a paradigm shift in universal models in computer vision.