An Evaluation of OCR on Egocentric Data

TL;DR

This paper evaluates current OCR methods on egocentric images, highlighting their struggles with rotated text and proposing a rotate-and-merge technique that improves accuracy, suggesting future models should incorporate rotation handling.

Contribution

The paper introduces a rotate-and-merge procedure that enhances OCR performance on rotated text in egocentric images, a novel approach for this application.

Findings

Existing OCR struggles with rotated text in egocentric data.

Rotate-and-merge halves the normalized edit distance error.

Rotation-aware training could further improve OCR accuracy.

Abstract

In this paper, we evaluate state-of-the-art OCR methods on Egocentric data. We annotate text in EPIC-KITCHENS images, and demonstrate that existing OCR methods struggle with rotated text, which is frequently observed on objects being handled. We introduce a simple rotate-and-merge procedure which can be applied to pre-trained OCR models that halves the normalized edit distance error. This suggests that future OCR attempts should incorporate rotation into model design and training procedures.