DEEP-CARVING: Discovering Visual Attributes by Carving Deep Neural Nets

TL;DR

This paper introduces Deep-Carving, a novel CNN training method that improves multi-attribute prediction in weakly supervised settings by iteratively carving features using pseudo-labels, and also presents a new dataset for attribute analysis.

Contribution

The paper proposes Deep-Carving, a new training procedure for CNNs that enhances multi-attribute prediction under weak supervision and introduces the CAMIT-NSAD dataset for visual attribute research.

Findings

Deep-Carving improves attribute prediction accuracy over baseline methods.

The method effectively disentangles co-occurring attributes in weakly supervised data.

Experiments on CAMIT-NSAD and SUN datasets validate the approach's effectiveness.

Abstract

Most of the approaches for discovering visual attributes in images demand significant supervision, which is cumbersome to obtain. In this paper, we aim to discover visual attributes in a weakly supervised setting that is commonly encountered with contemporary image search engines. Deep Convolutional Neural Networks (CNNs) have enjoyed remarkable success in vision applications recently. However, in a weakly supervised scenario, widely used CNN training procedures do not learn a robust model for predicting multiple attribute labels simultaneously. The primary reason is that the attributes highly co-occur within the training data. To ameliorate this limitation, we propose Deep-Carving, a novel training procedure with CNNs, that helps the net efficiently carve itself for the task of multiple attribute prediction. During training, the responses of the feature maps are exploited in an ingenious way to provide the net with multiple pseudo-labels (for training images) for subsequent iterations. The process is repeated periodically after a fixed number of iterations, and enables the net carve itself iteratively for efficiently disentangling features. Additionally, we contribute a noun-adjective pairing inspired Natural Scenes Attributes Dataset to the research community, CAMIT - NSAD, containing a number of co-occurring attributes within a noun category. We describe, in detail, salient aspects of this dataset. Our experiments on CAMIT-NSAD and the SUN Attributes Dataset, with weak supervision, clearly demonstrate that the Deep-Carved CNNs consistently achieve considerable improvement in the precision of attribute prediction over popular baseline methods.