# Object Recognition under Multifarious Conditions: A Reliability Analysis and A Feature Similarity-based Performance Estimation

## TL;DR

This study evaluates the reliability of popular online object recognition platforms under various real-world conditions and proposes a feature similarity framework to estimate their performance changes.

## Contribution

It introduces a novel framework combining handcrafted and deep learning features to predict recognition performance variations across different acquisition conditions.

## Key findings

- Deep neural network features correlate strongly with recognition performance (correlation 0.94).
- Recognition accuracy varies significantly with background, device, and object orientation.
- The proposed estimation method effectively predicts performance changes in diverse scenarios.

## Abstract

In this paper, we investigate the reliability of online recognition platforms, Amazon Rekognition and Microsoft Azure, with respect to changes in background, acquisition device, and object orientation. We focus on platforms that are commonly used by the public to better understand their real-world performances. To assess the variation in recognition performance, we perform a controlled experiment by changing the acquisition conditions one at a time. We use three smartphones, one DSLR, and one webcam to capture side views and overhead views of objects in a living room, an office, and photo studio setups. Moreover, we introduce a framework to estimate the recognition performance with respect to backgrounds and orientations. In this framework, we utilize both handcrafted features based on color, texture, and shape characteristics and data-driven features obtained from deep neural networks. Experimental results show that deep learning-based image representations can estimate the recognition performance variation with a Spearman's rank-order correlation of 0.94 under multifarious acquisition conditions.