Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection

TL;DR

This paper introduces a novel deep learning framework that uses box-level segmentation supervision to improve multispectral pedestrian detection accuracy and speed, especially for small and occluded pedestrians, suitable for real-time applications.

Contribution

It proposes a new segmentation supervised learning approach that overcomes hyperparameter issues of anchor box methods and enhances detection of small and occluded pedestrians.

Findings

Outperforms state-of-the-art methods in accuracy.

Achieves real-time detection speeds.

Improves detection of small and occluded pedestrians.

Abstract

Effective fusion of complementary information captured by multi-modal sensors (visible and infrared cameras) enables robust pedestrian detection under various surveillance situations (e.g. daytime and nighttime). In this paper, we present a novel box-level segmentation supervised learning framework for accurate and real-time multispectral pedestrian detection by incorporating features extracted in visible and infrared channels. Specifically, our method takes pairs of aligned visible and infrared images with easily obtained bounding box annotations as input and estimates accurate prediction maps to highlight the existence of pedestrians. It offers two major advantages over the existing anchor box based multispectral detection methods. Firstly, it overcomes the hyperparameter setting problem occurred during the training phase of anchor box based detectors and can obtain more accurate detection results, especially for small and occluded pedestrian instances. Secondly, it is capable of generating accurate detection results using small-size input images, leading to improvement of computational efficiency for real-time autonomous driving applications. Experimental results on KAIST multispectral dataset show that our proposed method outperforms state-of-the-art approaches in terms of both accuracy and speed.