Notes on image annotation

TL;DR

This paper discusses the challenges and insights gained from expert image annotation, highlighting the effort required for accurate scene understanding and presenting the SUN database as a result.

Contribution

It provides an in-depth account of the annotation process and difficulties faced, contributing a large, consistently labeled image dataset for research.

Findings

Annotation reveals complexities in scene understanding

Effort improves annotation consistency

SUN database created from expert annotations

Abstract

We are under the illusion that seeing is effortless, but frequently the visual system is lazy and makes us believe that we understand something when in fact we don't. Labeling a picture forces us to become aware of the difficulties underlying scene understanding. Suddenly, the act of seeing is not effortless anymore. We have to make an effort in order to understand parts of the picture that we neglected at first glance. In this report, an expert image annotator relates her experience on segmenting and labeling tens of thousands of images. During this process, the notes she took try to highlight the difficulties encountered, the solutions adopted, and the decisions made in order to get a consistent set of annotations. Those annotations constitute the SUN database.