Integrating Deep Features for Material Recognition

TL;DR

This paper introduces a method to integrate deep features from multiple CNNs trained on different datasets for improved material recognition, achieving state-of-the-art results and near-human accuracy on a new benchmark dataset.

Contribution

It presents a novel feature integration approach that combines multiple CNN representations for material recognition, along with a new dataset and transfer learning application.

Findings

Achieves state-of-the-art performance on benchmark datasets.

Introduces a new material dataset called EFMD.

Reaches near-human accuracy of 84.0% on FMD dataset.

Abstract

We propose a method for integration of features extracted using deep representations of Convolutional Neural Networks (CNNs) each of which is learned using a different image dataset of objects and materials for material recognition. Given a set of representations of multiple pre-trained CNNs, we first compute activations of features using the representations on the images to select a set of samples which are best represented by the features. Then, we measure the uncertainty of the features by computing the entropy of class distributions for each sample set. Finally, we compute the contribution of each feature to representation of classes for feature selection and integration. We examine the proposed method on three benchmark datasets for material recognition. Experimental results show that the proposed method achieves state-of-the-art performance by integrating deep features. Additionally, we introduce a new material dataset called EFMD by extending Flickr Material Database (FMD). By the employment of the EFMD with transfer learning for updating the learned CNN models, we achieve 84.0%+/-1.8% accuracy on the FMD dataset which is close to human performance that is 84.9%.