Vision-Based Human Awareness Estimation for Enhanced Safety and Efficiency of AMRs in Industrial Warehouses

TL;DR

This paper introduces a real-time vision-based system using a single RGB camera to estimate human awareness of AMRs in warehouses, improving safety and efficiency by enabling adaptive robot behavior.

Contribution

The novel integration of 3D human pose lifting and head orientation estimation for real-time awareness detection in industrial environments.

Findings

System reliably detects human positions and attention in real time.

Validation conducted using NVIDIA Isaac Sim with synthetic data.

Enables AMRs to adapt motion based on human awareness.

Abstract

Ensuring human safety is of paramount importance in warehouse environments that feature mixed traffic of human workers and autonomous mobile robots (AMRs). Current approaches often treat humans as generic dynamic obstacles, leading to conservative AMR behaviors like slowing down or detouring, even when workers are fully aware and capable of safely sharing space. This paper presents a real-time vision-based method to estimate human awareness of an AMR using a single RGB camera. We integrate state-of-the-art 3D human pose lifting with head orientation estimation to ascertain a human's position relative to the AMR and their viewing cone, thereby determining if the human is aware of the AMR. The entire pipeline is validated using synthetically generated data within NVIDIA Isaac Sim, a robust physics-accurate robotics simulation environment. Experimental results confirm that our system reliably detects human positions and their attention in real time, enabling AMRs to safely adapt their motion based on human awareness. This enhancement is crucial for improving both safety and operational efficiency in industrial and factory automation settings.