Posture recognition using an RGB-D camera : exploring 3D body modeling and deep learning approaches

TL;DR

This paper explores two novel supervised methods for human posture recognition using RGB-D cameras, employing deep learning on images and 3D skeleton modeling, achieving high accuracy and robustness across varied conditions.

Contribution

It introduces two new approaches for posture recognition with RGB-D data: CNN-based image analysis and 3D skeleton classification, demonstrating their effectiveness and robustness.

Findings

Both methods achieved high precision in posture recognition.

The CNN-based approach performed slightly better on depth images.

Methods showed robustness to scale and orientation variations.

Abstract

The emergence of RGB-D sensors offered new possibilities for addressing complex artificial vision problems efficiently. Human posture recognition is among these computer vision problems, with a wide range of applications such as ambient assisted living and intelligent health care systems. In this context, our paper presents novel methods and ideas to design automatic posture recognition systems using an RGB-D camera. More specifically, we introduce two supervised methods to learn and recognize human postures using the main types of visual data provided by an RGB-D camera. The first method is based on convolutional features extracted from 2D images. Convolutional Neural Networks (CNNs) are trained to recognize human postures using transfer learning on RGB and depth images. Secondly, we propose to model the posture using the body joint configuration in the 3D space. Posture recognition is then performed through SVM classification of 3D skeleton-based features. To evaluate the proposed methods, we created a challenging posture recognition dataset with a considerable variability regarding the acquisition conditions. The experimental results demonstrated comparable performances and high precision for both methods in recognizing human postures, with a slight superiority for the CNN-based method when applied on depth images. Moreover, the two approaches demonstrated a high robustness to several perturbation factors, such as scale and orientation change.