Detecting Humans in RGB-D Data with CNNs

TL;DR

This paper presents a novel approach for detecting humans in RGB-D data by combining CNNs for color and depth images, introducing a new depth-encoding scheme, and demonstrating improved performance on a public dataset.

Contribution

It introduces a new depth-encoding scheme and a fusion method for CNN-based human detection in RGB-D data, outperforming RGB-only models.

Findings

Depth-based CNN detection outperforms RGB-only models.

The new depth encoding enhances classification accuracy.

Fusion of color and depth CNNs improves detection results.

Abstract

We address the problem of people detection in RGB-D data where we leverage depth information to develop a region-of-interest (ROI) selection method that provides proposals to two color and depth CNNs. To combine the detections produced by the two CNNs, we propose a novel fusion approach based on the characteristics of depth images. We also present a new depth-encoding scheme, which not only encodes depth images into three channels but also enhances the information for classification. We conduct experiments on a publicly available RGB-D people dataset and show that our approach outperforms the baseline models that only use RGB data.