# TedEval: A Fair Evaluation Metric for Scene Text Detectors

## TL;DR

TedEval introduces a new evaluation metric for scene text detectors that assesses detection quality at the instance and character levels, addressing limitations of existing metrics and promoting fairer comparisons.

## Contribution

The paper proposes TedEval, a novel evaluation protocol that improves fairness and reliability in comparing scene text detection methods by considering recognition success.

## Key findings

- TedEval provides more accurate assessment of detection quality.
- It rewards behaviors leading to successful recognition.
- TedEval outperforms existing metrics in fairness and reliability.

## Abstract

Despite the recent success of scene text detection methods, common evaluation metrics fail to provide a fair and reliable comparison among detectors. They have obvious drawbacks in reflecting the inherent characteristic of text detection tasks, unable to address issues such as granularity, multiline, and character incompleteness. In this paper, we propose a novel evaluation protocol called TedEval (Text detector Evaluation), which evaluates text detections by an instance-level matching and a character-level scoring. Based on a firm standard rewarding behaviors that result in successful recognition, TedEval can act as a reliable standard for comparing and quantizing the detection quality throughout all difficulty levels. In this regard, we believe that TedEval can play a key role in developing state-of-the-art scene text detectors. The code is publicly available at https://github.com/clovaai/TedEval.