RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting

TL;DR

RAIN is a versatile motion forecasting framework that uses dynamic attention to select important information, achieving state-of-the-art results in multi-agent and human motion prediction tasks.

Contribution

The paper introduces RAIN, a novel hybrid attention-based framework with dynamic information selection and a double-stage training process for improved motion forecasting.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Provides interpretable attention weights for better understanding.

Effective in both multi-agent and human motion prediction tasks.

Abstract

Motion forecasting plays a significant role in various domains (e.g., autonomous driving, human-robot interaction), which aims to predict future motion sequences given a set of historical observations. However, the observed elements may be of different levels of importance. Some information may be irrelevant or even distracting to the forecasting in certain situations. To address this issue, we propose a generic motion forecasting framework (named RAIN) with dynamic key information selection and ranking based on a hybrid attention mechanism. The general framework is instantiated to handle multi-agent trajectory prediction and human motion forecasting tasks, respectively. In the former task, the model learns to recognize the relations between agents with a graph representation and to determine their relative significance. In the latter task, the model learns to capture the temporal proximity and dependency in long-term human motions. We also propose an effective double-stage training pipeline with an alternating training strategy to optimize the parameters in different modules of the framework. We validate the framework on both synthetic simulations and motion forecasting benchmarks in different domains, demonstrating that our method not only achieves state-of-the-art forecasting performance, but also provides interpretable and reasonable hybrid attention weights.