Are Elephants Bigger than Butterflies? Reasoning about Sizes of Objects

TL;DR

This paper presents a method to automatically infer and reason about object sizes in AI using visual and textual web data, creating a new dataset and outperforming baselines in size comparison tasks.

Contribution

The paper introduces a novel approach to automatically estimate object sizes without human supervision and provides a new dataset for size reasoning.

Findings

Outperforms baseline methods in size comparison accuracy

Creates a comprehensive relative size dataset

Demonstrates the importance of size information in visual reasoning

Abstract

Human vision greatly benefits from the information about sizes of objects. The role of size in several visual reasoning tasks has been thoroughly explored in human perception and cognition. However, the impact of the information about sizes of objects is yet to be determined in AI. We postulate that this is mainly attributed to the lack of a comprehensive repository of size information. In this paper, we introduce a method to automatically infer object sizes, leveraging visual and textual information from web. By maximizing the joint likelihood of textual and visual observations, our method learns reliable relative size estimates, with no explicit human supervision. We introduce the relative size dataset and show that our method outperforms competitive textual and visual baselines in reasoning about size comparisons.