Fused Deep Neural Networks for Efficient Pedestrian Detection

TL;DR

This paper introduces a fusion-based deep neural network system for pedestrian detection that combines multiple DNNs, a novel soft-rejection fusion method, and context-aware segmentation to achieve state-of-the-art accuracy and speed.

Contribution

The paper proposes a novel fusion framework with soft-rejection and soft-label training methods, improving pedestrian detection accuracy and efficiency over existing approaches.

Findings

Achieved a log-average miss rate of 7.67% on Caltech dataset.

Outperformed previous state-of-the-art methods in accuracy.

Maintained high detection speed while improving accuracy.

Abstract

In this paper, we present an efficient pedestrian detection system, designed by fusion of multiple deep neural network (DNN) systems. Pedestrian candidates are first generated by a single shot convolutional multi-box detector at different locations with various scales and aspect ratios. The candidate generator is designed to provide the majority of ground truth pedestrian annotations at the cost of a large number of false positives. Then, a classification system using the idea of ensemble learning is deployed to improve the detection accuracy. The classification system further classifies the generated candidates based on opinions of multiple deep verification networks and a fusion network which utilizes a novel soft-rejection fusion method to adjust the confidence in the detection results. To improve the training of the deep verification networks, a novel soft-label method is devised to assign floating point labels to the generated pedestrian candidates. A deep context aggregation semantic segmentation network also provides pixel-level classification of the scene and its results are softly fused with the detection results by the single shot detector. Our pedestrian detector compared favorably to state-of-art methods on all popular pedestrian detection datasets. For example, our fused DNN has better detection accuracy on the Caltech Pedestrian dataset than all previous state of art methods, while also being the fastest. We significantly improved the log-average miss rate on the Caltech pedestrian dataset to 7.67% and achieved the new state-of-the-art.