Robust Human Motion Forecasting using Transformer-based Model

TL;DR

This paper introduces a lightweight, robust Transformer-based model for real-time 3D human motion forecasting that outperforms existing models in accuracy and efficiency, especially under occlusion and noisy conditions.

Contribution

The proposed 2-Channel Transformer (2CH-TR) is a novel model that effectively exploits spatio-temporal information for short and long-term human motion prediction, with improved robustness and speed.

Findings

Outperforms ST-Transformer in accuracy

Reduces mean squared error by 8.89% short-term

Operates efficiently in noisy, occluded environments

Abstract

Comprehending human motion is a fundamental challenge for developing Human-Robot Collaborative applications. Computer vision researchers have addressed this field by only focusing on reducing error in predictions, but not taking into account the requirements to facilitate its implementation in robots. In this paper, we propose a new model based on Transformer that simultaneously deals with the real time 3D human motion forecasting in the short and long term. Our 2-Channel Transformer (2CH-TR) is able to efficiently exploit the spatio-temporal information of a shortly observed sequence (400ms) and generates a competitive accuracy against the current state-of-the-art. 2CH-TR stands out for the efficient performance of the Transformer, being lighter and faster than its competitors. In addition, our model is tested in conditions where the human motion is severely occluded, demonstrating its robustness in reconstructing and predicting 3D human motion in a highly noisy environment. Our experiment results show that the proposed 2CH-TR outperforms the ST-Transformer, which is another state-of-the-art model based on the Transformer, in terms of reconstruction and prediction under the same conditions of input prefix. Our model reduces in 8.89% the mean squared error of ST-Transformer in short-term prediction, and 2.57% in long-term prediction in Human3.6M dataset with 400ms input prefix. Webpage: https://evm7.github.io/2CHTR-page/