Recognizing Actions from Robotic View for Natural Human-Robot Interaction

TL;DR

This paper introduces ACTIVE, a large-scale dataset for recognizing human actions from robotic views in natural human-robot interaction, and proposes ACTIVE-PC, a method for accurate long-distance action perception in dynamic environments.

Contribution

The paper presents a new comprehensive dataset ACTIVE tailored for robotic view action recognition and introduces ACTIVE-PC, a novel method for perceiving actions at long distances in mobile robot scenarios.

Findings

ACTIVE dataset covers diverse environments and modalities.

ACTIVE-PC achieves accurate long-distance action recognition.

Experimental results validate the effectiveness of ACTIVE-PC.

Abstract

Natural Human-Robot Interaction (N-HRI) requires robots to recognize human actions at varying distances and states, regardless of whether the robot itself is in motion or stationary. This setup is more flexible and practical than conventional human action recognition tasks. However, existing benchmarks designed for traditional action recognition fail to address the unique complexities in N-HRI due to limited data, modalities, task categories, and diversity of subjects and environments. To address these challenges, we introduce ACTIVE (Action from Robotic View), a large-scale dataset tailored specifically for perception-centric robotic views prevalent in mobile service robots. ACTIVE comprises 30 composite action categories, 80 participants, and 46,868 annotated video instances, covering both RGB and point cloud modalities. Participants performed various human actions in diverse environments at distances ranging from 3m to 50m, while the camera platform was also mobile, simulating real-world scenarios of robot perception with varying camera heights due to uneven ground. This comprehensive and challenging benchmark aims to advance action and attribute recognition research in N-HRI. Furthermore, we propose ACTIVE-PC, a method that accurately perceives human actions at long distances using Multilevel Neighborhood Sampling, Layered Recognizers, Elastic Ellipse Query, and precise decoupling of kinematic interference from human actions. Experimental results demonstrate the effectiveness of ACTIVE-PC. Our code is available at: https://github.com/wangzy01/ACTIVE-Action-from-Robotic-View.