Freehand Sketch Recognition Using Deep Features

TL;DR

This paper presents a novel framework for freehand sketch recognition using deep features extracted from CNNs, achieving improved accuracy and potential for related applications like sketch-based image retrieval.

Contribution

It introduces a CNN-based sketch recognition method that outperforms previous approaches and explores the use of deep features for attribute identification.

Findings

Recognition accuracy improved by 3%-11% over state-of-the-art.

Deep features are effective for sketch recognition and retrieval.

Preliminary analysis shows potential for attribute identification.

Abstract

Freehand sketches often contain sparse visual detail. In spite of the sparsity, they are easily and consistently recognized by humans across cultures, languages and age groups. Therefore, analyzing such sparse sketches can aid our understanding of the neuro-cognitive processes involved in visual representation and recognition. In the recent past, Convolutional Neural Networks (CNNs) have emerged as a powerful framework for feature representation and recognition for a variety of image domains. However, the domain of sketch images has not been explored. This paper introduces a freehand sketch recognition framework based on "deep" features extracted from CNNs. We use two popular CNNs for our experiments -- Imagenet CNN and a modified version of LeNet CNN. We evaluate our recognition framework on a publicly available benchmark database containing thousands of freehand sketches depicting everyday objects. Our results are an improvement over the existing state-of-the-art accuracies by 3% - 11%. The effectiveness and relative compactness of our deep features also make them an ideal candidate for related problems such as sketch-based image retrieval. In addition, we provide a preliminary glimpse of how such features can help identify crucial attributes (e.g. object-parts) of the sketched objects.