Unsupervised Semantic-based Aggregation of Deep Convolutional Features

TL;DR

This paper introduces an unsupervised semantic-based aggregation method for deep convolutional features that enhances image representation by selecting discriminative semantic detectors, leading to improved performance across multiple visual tasks.

Contribution

It presents a novel unsupervised strategy to select semantic detectors for aggregating deep features, which is simple, effective, and generalizes well to various applications.

Findings

Outperforms state-of-the-art aggregation methods in image retrieval.

Achieves high accuracy in place recognition.

Effective in cloud classification tasks.

Abstract

In this paper, we propose a simple but effective semantic-based aggregation (SBA) method. The proposed SBA utilizes the discriminative filters of deep convolutional layers as semantic detectors. Moreover, we propose the effective unsupervised strategy to select some semantic detectors to generate the "probabilistic proposals", which highlight certain discriminative pattern of objects and suppress the noise of background. The final global SBA representation could then be acquired by aggregating the regional representations weighted by the selected "probabilistic proposals" corresponding to various semantic content. Our unsupervised SBA is easy to generalize and achieves excellent performance on various tasks. We conduct comprehensive experiments and show that our unsupervised SBA outperforms the state-of-the-art unsupervised and supervised aggregation methods on image retrieval, place recognition and cloud classification.