CURE-OR: Challenging Unreal and Real Environments for Object Recognition

TL;DR

This paper introduces CURE-OR, a large-scale dataset with challenging real and unreal images for object recognition, revealing how recognition APIs perform under various difficult conditions and exploring the link between image quality and recognition accuracy.

Contribution

The paper presents a new extensive dataset, CURE-OR, and analyzes recognition API performance under diverse challenging conditions, also linking image quality metrics to recognition success.

Findings

Recognition APIs' performance drops significantly under challenging conditions.

Objective quality algorithms can predict recognition performance in certain scenarios.

CURE-OR dataset is publicly available for further research.

Abstract

In this paper, we introduce a large-scale, controlled, and multi-platform object recognition dataset denoted as Challenging Unreal and Real Environments for Object Recognition (CURE-OR). In this dataset, there are 1,000,000 images of 100 objects with varying size, color, and texture that are positioned in five different orientations and captured using five devices including a webcam, a DSLR, and three smartphone cameras in real-world (real) and studio (unreal) environments. The controlled challenging conditions include underexposure, overexposure, blur, contrast, dirty lens, image noise, resizing, and loss of color information. We utilize CURE-OR dataset to test recognition APIs-Amazon Rekognition and Microsoft Azure Computer Vision- and show that their performance significantly degrades under challenging conditions. Moreover, we investigate the relationship between object recognition and image quality and show that objective quality algorithms can estimate recognition performance under certain photometric challenging conditions. The dataset is publicly available at https://ghassanalregib.com/cure-or/.