# Informative Object Annotations: Tell Me Something I Don't Know

## TL;DR

This paper introduces an unsupervised method for selecting informative image labels based on prior knowledge, using entropy reduction to improve image annotation relevance and aligning well with human judgments.

## Contribution

It presents a novel unsupervised approach to model prior knowledge and quantify label informativeness for image annotation, inspired by cognitive theories.

## Key findings

- Achieves ~65% agreement with human raters
- Outperforms other unsupervised baseline methods
- Efficient approximation of entropy reduction using graphical models

## Abstract

Capturing the interesting components of an image is a key aspect of image understanding. When a speaker annotates an image, selecting labels that are informative greatly depends on the prior knowledge of a prospective listener. Motivated by cognitive theories of categorization and communication, we present a new unsupervised approach to model this prior knowledge and quantify the informativeness of a description. Specifically, we compute how knowledge of a label reduces uncertainty over the space of labels and utilize this to rank candidate labels for describing an image. While the full estimation problem is intractable, we describe an efficient algorithm to approximate entropy reduction using a tree-structured graphical model. We evaluate our approach on the open-images dataset using a new evaluation set of 10K ground-truth ratings and find that it achieves ~65% agreement with human raters, largely outperforming other unsupervised baseline approaches.