Learning Deep Context-Network Architectures for Image Annotation

TL;DR

This paper introduces a deep learning framework that learns spatial context for image annotation, resulting in more effective kernels for image classification, demonstrated on the ImageCLEF benchmark.

Contribution

It presents a novel deep network architecture that learns spatial context discriminatively for kernel design in image annotation tasks.

Findings

Deep context learning improves image classification accuracy.

The proposed method outperforms traditional handcrafted context approaches.

Extensive experiments validate the effectiveness of the learned kernels.

Abstract

Context plays an important role in visual pattern recognition as it provides complementary clues for different learning tasks including image classification and annotation. In the particular scenario of kernel learning, the general recipe of context-based kernel design consists in learning positive semi-definite similarity functions that return high values not only when data share similar content but also similar context. However, in spite of having a positive impact on performance, the use of context in these kernel design methods has not been fully explored; indeed, context has been handcrafted instead of being learned. In this paper, we introduce a novel context-aware kernel design framework based on deep learning. Our method discriminatively learns spatial geometric context as the weights of a deep network (DN). The architecture of this network is fully determined by the solution of an objective function that mixes content, context and regularization, while the parameters of this network determine the most relevant (discriminant) parts of the learned context. We apply this context and kernel learning framework to image classification using the challenging ImageCLEF Photo Annotation benchmark; the latter shows that our deep context learning provides highly effective kernels for image classification as corroborated through extensive experiments.