TL;DR
This paper introduces FS-COCO, a new dataset of 10,000 freehand scene sketches with descriptions, enabling research on scene understanding, image retrieval, and multimodal analysis with practical applications.
Contribution
The paper presents the first dataset of freehand scene sketches with annotations and explores novel tasks like fine-grained image retrieval and sketch-caption analysis.
Findings
Sketch strokes encode scene salience via temporal order.
Image retrieval performance varies between sketches and captions.
Combining sketches and captions improves retrieval accuracy.
Abstract
We advance sketch research to scenes with the first dataset of freehand scene sketches, FS-COCO. With practical applications in mind, we collect sketches that convey scene content well but can be sketched within a few minutes by a person with any sketching skills. Our dataset comprises 10,000 freehand scene vector sketches with per point space-time information by 100 non-expert individuals, offering both object- and scene-level abstraction. Each sketch is augmented with its text description. Using our dataset, we study for the first time the problem of fine-grained image retrieval from freehand scene sketches and sketch captions. We draw insights on: (i) Scene salience encoded in sketches using the strokes temporal order; (ii) Performance comparison of image retrieval from a scene sketch and an image caption; (iii) Complementarity of information in sketches and image captions, as well…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
