Advancing Generative Model Evaluation: A Novel Algorithm for Realistic Image Synthesis and Comparison in OCR System

TL;DR

This paper introduces a novel algorithm to objectively evaluate and compare the realism of synthetic images generated by models, specifically improving assessment methods for Arabic handwritten digits in OCR systems.

Contribution

A new algorithm that refines the FID score for more accurate and subjective evaluation of realistic image synthesis, especially for complex scripts like Arabic.

Findings

Enhanced FID score for better realism assessment

Systematic comparison of generative models for Arabic digits

Improved evaluation framework for OCR image quality

Abstract

This research addresses a critical challenge in the field of generative models, particularly in the generation and evaluation of synthetic images. Given the inherent complexity of generative models and the absence of a standardized procedure for their comparison, our study introduces a pioneering algorithm to objectively assess the realism of synthetic images. This approach significantly enhances the evaluation methodology by refining the Fr\'echet Inception Distance (FID) score, allowing for a more precise and subjective assessment of image quality. Our algorithm is particularly tailored to address the challenges in generating and evaluating realistic images of Arabic handwritten digits, a task that has traditionally been near-impossible due to the subjective nature of realism in image generation. By providing a systematic and objective framework, our method not only enables the comparison of different generative models but also paves the way for improvements in their design and output. This breakthrough in evaluation and comparison is crucial for advancing the field of OCR, especially for scripts that present unique complexities, and sets a new standard in the generation and assessment of high-quality synthetic images.