3D Pose Nowcasting: Forecast the Future to Improve the Present

TL;DR

This paper introduces a vision-based system that uses depth data to estimate 3D human and robot poses more accurately by forecasting future poses, thereby improving current pose estimation in real-time applications.

Contribution

It presents the novel concept of Pose Nowcasting, combining pose forecasting with current estimation to enhance accuracy in human-robot interaction scenarios.

Findings

Accurate real-time 3D pose estimation demonstrated on two datasets.

Forecasting future poses improves current pose accuracy.

Validates effectiveness in both robotic and human contexts.

Abstract

Technologies to enable safe and effective collaboration and coexistence between humans and robots have gained significant importance in the last few years. A critical component useful for realizing this collaborative paradigm is the understanding of human and robot 3D poses using non-invasive systems. Therefore, in this paper, we propose a novel vision-based system leveraging depth data to accurately establish the 3D locations of skeleton joints. Specifically, we introduce the concept of Pose Nowcasting, denoting the capability of the proposed system to enhance its current pose estimation accuracy by jointly learning to forecast future poses. The experimental evaluation is conducted on two different datasets, providing accurate and real-time performance and confirming the validity of the proposed method on both the robotic and human scenarios.