Label or Message: A Large-Scale Experimental Survey of Texts and Objects Co-Occurrence

TL;DR

This paper presents a large-scale survey analyzing the co-occurrence of visual objects and scene texts, focusing on label texts attached to objects, to understand their mutual usefulness for recognition tasks.

Contribution

It introduces a comprehensive analysis of object-text co-occurrence using a large dataset and state-of-the-art text recognition, highlighting the role of label texts in scene understanding.

Findings

Objects and texts frequently co-occur in scenes.

Label texts provide valuable information for object recognition.

Scene texts can enhance the accuracy of visual object detection.

Abstract

Our daily life is surrounded by textual information. Nowadays, the automatic collection of textual information becomes possible owing to the drastic improvement of scene text detectors and recognizer. The purpose of this paper is to conduct a large-scale survey of co-occurrence between visual objects (such as book and car) and scene texts with a large image dataset and a state-of-the-art scene text detector and recognizer. Especially, we focus on the function of "label" texts, which are attached to objects for detailing the objects. By analyzing co-occurrence between objects and scene texts, it is possible to observe the statistics about the label texts and understand how the scene texts will be useful for recognizing the objects and vice versa.