Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reach

TL;DR

This paper introduces a large dataset of gaze-primed human motions and a diffusion-based model that generates realistic, goal-oriented full-body movements, effectively mimicking natural priming and reaching behaviors.

Contribution

It is the first to curate a large dataset of gaze-primed motions and to develop a goal-conditioned diffusion model for realistic human motion synthesis.

Findings

Model outperforms baselines on multiple datasets

Generated motions exhibit natural priming and reaching behaviors

Introduces new 'Prime Success' metric for evaluation

Abstract

Human motion generation is a challenging task that aims to create realistic motion imitating natural human behaviour. We focus on the well-studied behaviour of priming an object/location for pick up or put down - that is, the spotting of an object/location from a distance, known as gaze priming, followed by the motion of approaching and reaching the target location. To that end, we curate, for the first time, 23.7K gaze-primed human motion sequences for reaching target object locations from five publicly available datasets, i.e., HD-EPIC, MoGaze, HOT3D, ADT, and GIMO. We pre-train a text-conditioned diffusion-based motion generation model, then fine-tune it conditioned on goal pose or location, on our curated sequences. Importantly, we evaluate the ability of the generated motion to imitate natural human movement through several metrics, including the 'Reach Success' and a newly introduced 'Prime Success' metric. Tested on 5 datasets, our model generates diverse full-body motion, exhibiting both priming and reaching behaviour, and outperforming baselines and recent methods.