Scene Image Representation by Foreground, Background and Hybrid Features

TL;DR

This paper introduces a hybrid feature approach combining foreground, background, and hybrid information from deep learning models to improve scene image classification accuracy, outperforming existing methods on benchmark datasets.

Contribution

The paper proposes a novel hybrid feature extraction method using three VGG-16 models for improved scene image representation and classification.

Findings

Achieved state-of-the-art accuracy on MIT-67 dataset.

Achieved state-of-the-art accuracy on SUN-397 dataset.

Demonstrated the effectiveness of combining foreground, background, and hybrid features.

Abstract

Previous methods for representing scene images based on deep learning primarily consider either the foreground or background information as the discriminating clues for the classification task. However, scene images also require additional information (hybrid) to cope with the inter-class similarity and intra-class variation problems. In this paper, we propose to use hybrid features in addition to foreground and background features to represent scene images. We suppose that these three types of information could jointly help to represent scene image more accurately. To this end, we adopt three VGG-16 architectures pre-trained on ImageNet, Places, and Hybrid (both ImageNet and Places) datasets for the corresponding extraction of foreground, background and hybrid information. All these three types of deep features are further aggregated to achieve our final features for the representation of scene images. Extensive experiments on two large benchmark scene datasets (MIT-67 and SUN-397) show that our method produces the state-of-the-art classification performance.