Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion

TL;DR

This paper introduces a novel deep architecture called Compositional CNN that inherently improves robustness to partial occlusion by integrating compositional models, enabling better classification and occluder localization without extensive occlusion-specific training.

Contribution

The paper proposes replacing the fully connected classification head of a DCNN with a differentiable compositional model, enhancing robustness to partial occlusion and enabling occluder localization.

Findings

Outperforms standard DCNNs on occluded object classification

Can localize occluders accurately with class labels only

Maintains high accuracy even without occlusion-specific training

Abstract

Recent findings show that deep convolutional neural networks (DCNNs) do not generalize well under partial occlusion. Inspired by the success of compositional models at classifying partially occluded objects, we propose to integrate compositional models and DCNNs into a unified deep model with innate robustness to partial occlusion. We term this architecture Compositional Convolutional Neural Network. In particular, we propose to replace the fully connected classification head of a DCNN with a differentiable compositional model. The generative nature of the compositional model enables it to localize occluders and subsequently focus on the non-occluded parts of the object. We conduct classification experiments on artificially occluded images as well as real images of partially occluded objects from the MS-COCO dataset. The results show that DCNNs do not classify occluded objects robustly, even when trained with data that is strongly augmented with partial occlusions. Our proposed model outperforms standard DCNNs by a large margin at classifying partially occluded objects, even when it has not been exposed to occluded objects during training. Additional experiments demonstrate that CompositionalNets can also localize the occluders accurately, despite being trained with class labels only. The code used in this work is publicly available.