Deep Learning for Free-Hand Sketch: A Survey

TL;DR

This survey comprehensively reviews deep learning techniques applied to free-hand sketch data, highlighting unique challenges, datasets, methods, and future research directions in sketch recognition and generation.

Contribution

It provides a detailed taxonomy and experimental evaluation of deep learning approaches for free-hand sketch analysis, emphasizing intrinsic traits and research challenges.

Findings

Identified key datasets and research topics in sketch deep learning

Analyzed state-of-the-art methods and their performance

Discussed bottlenecks and open problems for future research

Abstract

Free-hand sketches are highly illustrative, and have been widely used by humans to depict objects or stories from ancient times to the present. The recent prevalence of touchscreen devices has made sketch creation a much easier task than ever and consequently made sketch-oriented applications increasingly popular. The progress of deep learning has immensely benefited free-hand sketch research and applications. This paper presents a comprehensive survey of the deep learning techniques oriented at free-hand sketch data, and the applications that they enable. The main contents of this survey include: (i) A discussion of the intrinsic traits and unique challenges of free-hand sketch, to highlight the essential differences between sketch data and other data modalities, e.g., natural photos. (ii) A review of the developments of free-hand sketch research in the deep learning era, by surveying existing datasets, research topics, and the state-of-the-art methods through a detailed taxonomy and experimental evaluation. (iii) Promotion of future work via a discussion of bottlenecks, open problems, and potential research directions for the community.