Data-Driven Stochastic Motion Evaluation and Optimization with Image by Spatially-Aligned Temporal Encoding

TL;DR

This paper introduces a probabilistic motion prediction framework that integrates image and motion data using spatially-aligned temporal encoding, coupled with a self-supervised optimizer to improve long motion task performance.

Contribution

It presents a novel integration of image and motion data via spatially-aligned temporal encoding and a self-supervised Deep Motion Optimizer for improved motion prediction.

Findings

Effective motion prediction demonstrated on various tasks.

Outperforms similar state-of-the-art methods.

Reduces hyper-parameter tuning difficulties.

Abstract

This paper proposes a probabilistic motion prediction method for long motions. The motion is predicted so that it accomplishes a task from the initial state observed in the given image. While our method evaluates the task achievability by the Energy-Based Model (EBM), previous EBMs are not designed for evaluating the consistency between different domains (i.e., image and motion in our method). Our method seamlessly integrates the image and motion data into the image feature domain by spatially-aligned temporal encoding so that features are extracted along the motion trajectory projected onto the image. Furthermore, this paper also proposes a data-driven motion optimization method, Deep Motion Optimizer (DMO), that works with EBM for motion prediction. Different from previous gradient-based optimizers, our self-supervised DMO alleviates the difficulty of hyper-parameter tuning to avoid local minima. The effectiveness of the proposed method is demonstrated with a variety of experiments with similar SOTA methods.