Few-shot human motion prediction for heterogeneous sensors

TL;DR

This paper introduces a novel few-shot human motion prediction method that explicitly incorporates sensor graphs, enabling better generalization across diverse sensor setups and outperforming existing models significantly.

Contribution

It presents the first few-shot motion prediction approach using graph neural networks that handles heterogeneous sensors and generalizes across different motion tasks.

Findings

Achieved 10.4% to 39.3% performance improvements over state-of-the-art.

Performs comparably to best models on fixed sensor tasks with fewer parameters.

Demonstrates effective generalization across heterogeneous sensor configurations.

Abstract

Human motion prediction is a complex task as it involves forecasting variables over time on a graph of connected sensors. This is especially true in the case of few-shot learning, where we strive to forecast motion sequences for previously unseen actions based on only a few examples. Despite this, almost all related approaches for few-shot motion prediction do not incorporate the underlying graph, while it is a common component in classical motion prediction. Furthermore, state-of-the-art methods for few-shot motion prediction are restricted to motion tasks with a fixed output space meaning these tasks are all limited to the same sensor graph. In this work, we propose to extend recent works on few-shot time-series forecasting with heterogeneous attributes with graph neural networks to introduce the first few-shot motion approach that explicitly incorporates the spatial graph while also generalizing across motion tasks with heterogeneous sensors. In our experiments on motion tasks with heterogeneous sensors, we demonstrate significant performance improvements with lifts from 10.4% up to 39.3% compared to best state-of-the-art models. Moreover, we show that our model can perform on par with the best approach so far when evaluating on tasks with a fixed output space while maintaining two magnitudes fewer parameters.