Predicting Human Intentions from Motion Only: A 2D+3D Fusion Approach

TL;DR

This paper introduces a multimodal 2D+3D fusion approach to predict human intentions solely from motion data, demonstrating that combining these modalities improves accuracy in a context-free setting.

Contribution

It presents a new dataset and experimental framework for intent prediction from motion, and develops a multimodal fusion method that outperforms single-modality approaches.

Findings

Multimodal 2D+3D fusion improves intent prediction accuracy.

Video data alone yields lower performance but adds complementary information.

The approach predicts future actions without contextual cues.

Abstract

In this paper, we address the new problem of the prediction of human intents. There is neuro-psychological evidence that actions performed by humans are anticipated by peculiar motor acts which are discriminant of the type of action going to be performed afterwards. In other words, an actual intent can be forecast by looking at the kinematics of the immediately preceding movement. To prove it in a computational and quantitative manner, we devise a new experimental setup where, without using contextual information, we predict human intents all originating from the same motor act. We posit the problem as a classification task and we introduce a new multi-modal dataset consisting of a set of motion capture marker 3D data and 2D video sequences, where, by only analysing very similar movements in both training and test phases, we are able to predict the underlying intent, i.e., the future, never observed action. We also present an extensive experimental evaluation as a baseline, customizing state-of-the-art techniques for either 3D and 2D data analysis. Realizing that video processing methods lead to inferior performance but show complementary information with respect to 3D data sequences, we developed a 2D+3D fusion analysis where we achieve better classification accuracies, attesting the superiority of the multimodal approach for the context-free prediction of human intents.