Deep Feature Augmentation for Occluded Image Classification

TL;DR

This paper introduces a deep feature augmentation method that enhances occluded image classification accuracy by fine-tuning pre-trained CNNs with augmented feature vectors, including pseudo-DFVs generated from small datasets, showing significant improvements.

Contribution

The paper proposes a novel deep feature augmentation technique using pseudo-DFVs to improve occluded image classification without large datasets, and demonstrates its effectiveness across datasets and models.

Findings

Achieves over 11% accuracy improvement on ILSVRC2012 with synthetic occlusions.

Effective across various datasets and network architectures.

Does not significantly affect performance on clean images.

Abstract

Due to the difficulty in acquiring massive task-specific occluded images, the classification of occluded images with deep convolutional neural networks (CNNs) remains highly challenging. To alleviate the dependency on large-scale occluded image datasets, we propose a novel approach to improve the classification accuracy of occluded images by fine-tuning the pre-trained models with a set of augmented deep feature vectors (DFVs). The set of augmented DFVs is composed of original DFVs and pseudo-DFVs. The pseudo-DFVs are generated by randomly adding difference vectors (DVs), extracted from a small set of clean and occluded image pairs, to the real DFVs. In the fine-tuning, the back-propagation is conducted on the DFV data flow to update the network parameters. The experiments on various datasets and network structures show that the deep feature augmentation significantly improves the classification accuracy of occluded images without a noticeable influence on the performance of clean images. Specifically, on the ILSVRC2012 dataset with synthetic occluded images, the proposed approach achieves 11.21% and 9.14% average increases in classification accuracy for the ResNet50 networks fine-tuned on the occlusion-exclusive and occlusion-inclusive training sets, respectively.