Rosetta: Large scale system for text detection and recognition in images

TL;DR

Rosetta is a scalable OCR system deployed at Facebook that efficiently detects and recognizes text in images, enabling improved media understanding for social media search and recommendation.

Contribution

The paper introduces a large-scale, practical OCR system architecture and modeling techniques tailored for processing billions of images daily at Facebook.

Findings

Effective detection and recognition models for large-scale deployment

Practical approaches for building scalable OCR systems

Insights into system components and their performance

Abstract

In this paper we present a deployed, scalable optical character recognition (OCR) system, which we call Rosetta, designed to process images uploaded daily at Facebook scale. Sharing of image content has become one of the primary ways to communicate information among internet users within social networks such as Facebook and Instagram, and the understanding of such media, including its textual information, is of paramount importance to facilitate search and recommendation applications. We present modeling techniques for efficient detection and recognition of text in images and describe Rosetta's system architecture. We perform extensive evaluation of presented technologies, explain useful practical approaches to build an OCR system at scale, and provide insightful intuitions as to why and how certain components work based on the lessons learnt during the development and deployment of the system.