QUB-PHEO: A Visual-Based Dyadic Multi-View Dataset for Intention Inference in Collaborative Assembly

TL;DR

QUB-PHEO is a comprehensive visual dataset capturing multimodal interactions between humans during assembly tasks, aimed at advancing intention inference and human-robot collaboration research.

Contribution

It provides a novel, richly annotated dyadic dataset with visual cues for intention inference in assembly, supporting improved machine learning models in HRI.

Findings

Rich multimodal data for human-robot interaction

Detailed visual annotations including facial landmarks and gaze

Supports development of intention inference algorithms

Abstract

QUB-PHEO introduces a visual-based, dyadic dataset with the potential of advancing human-robot interaction (HRI) research in assembly operations and intention inference. This dataset captures rich multimodal interactions between two participants, one acting as a 'robot surrogate,' across a variety of assembly tasks that are further broken down into 36 distinct subtasks. With rich visual annotations, such as facial landmarks, gaze, hand movements, object localization, and more for 70 participants, QUB-PHEO offers two versions: full video data for 50 participants and visual cues for all 70. Designed to improve machine learning models for HRI, QUB-PHEO enables deeper analysis of subtle interaction cues and intentions, promising contributions to the field. The dataset will be available at https://github.com/exponentialR/QUB-PHEO subject to an End-User License Agreement (EULA).