Deformable Part Models are Convolutional Neural Networks

TL;DR

This paper demonstrates that deformable part models can be reformulated as convolutional neural networks, enabling the integration of learned features and leading to improved detection performance and efficiency.

Contribution

It provides a novel formulation unifying DPMs and CNNs by unrolling DPM inference into CNN layers, allowing learned features to replace traditional image features.

Findings

DeepPyramid DPM outperforms traditional HOG-based DPMs.

It slightly surpasses R-CNN in detection accuracy.

The approach runs an order of magnitude faster than R-CNN.

Abstract

Deformable part models (DPMs) and convolutional neural networks (CNNs) are two widely used tools for visual recognition. They are typically viewed as distinct approaches: DPMs are graphical models (Markov random fields), while CNNs are "black-box" non-linear classifiers. In this paper, we show that a DPM can be formulated as a CNN, thus providing a novel synthesis of the two ideas. Our construction involves unrolling the DPM inference algorithm and mapping each step to an equivalent (and at times novel) CNN layer. From this perspective, it becomes natural to replace the standard image features used in DPM with a learned feature extractor. We call the resulting model DeepPyramid DPM and experimentally validate it on PASCAL VOC. DeepPyramid DPM significantly outperforms DPMs based on histograms of oriented gradients features (HOG) and slightly outperforms a comparable version of the recently introduced R-CNN detection system, while running an order of magnitude faster.