What Will I Do Next? The Intention from Motion Experiment

TL;DR

This study demonstrates that video-based approaches can reliably predict human intentions from grasping actions, matching the performance of 3D kinematic methods, thus bridging cognitive insights and computer vision techniques.

Contribution

The paper introduces the 'Intention from Motion' paradigm, showing video-based methods can predict intentions without contextual info, comparable to 3D kinematic analysis.

Findings

Video-based techniques achieve equivalent performance to 3D kinematic descriptors.

Computer vision tools effectively capture kinematic cues for intention prediction.

The approach predicts intentions from grasping onset without contextual information.

Abstract

In computer vision, video-based approaches have been widely explored for the early classification and the prediction of actions or activities. However, it remains unclear whether this modality (as compared to 3D kinematics) can still be reliable for the prediction of human intentions, defined as the overarching goal embedded in an action sequence. Since the same action can be performed with different intentions, this problem is more challenging but yet affordable as proved by quantitative cognitive studies which exploit the 3D kinematics acquired through motion capture systems. In this paper, we bridge cognitive and computer vision studies, by demonstrating the effectiveness of video-based approaches for the prediction of human intentions. Precisely, we propose Intention from Motion, a new paradigm where, without using any contextual information, we consider instantaneous grasping motor acts involving a bottle in order to forecast why the bottle itself has been reached (to pass it or to place in a box, or to pour or to drink the liquid inside). We process only the grasping onsets casting intention prediction as a classification framework. Leveraging on our multimodal acquisition (3D motion capture data and 2D optical videos), we compare the most commonly used 3D descriptors from cognitive studies with state-of-the-art video-based techniques. Since the two analyses achieve an equivalent performance, we demonstrate that computer vision tools are effective in capturing the kinematics and facing the cognitive problem of human intention prediction.