On the Minimal Recognizable Image Patch

TL;DR

This paper introduces the concept of minimal recognizable patches (MRPs) to empirically explore the limits of recognition algorithms, revealing sharp declines in accuracy with size reduction similar to human perception.

Contribution

It proposes a method using a specialized deep network to identify the most informative image patches for recognition, establishing computational analogues to human minimal recognizable configurations.

Findings

Recognition accuracy drops sharply with patch size reduction.

Deep network effectively identifies minimal informative patches.

Computational minimal recognizable patches mirror human perception patterns.

Abstract

In contrast to human vision, common recognition algorithms often fail on partially occluded images. We propose characterizing, empirically, the algorithmic limits by finding a minimal recognizable patch (MRP) that is by itself sufficient to recognize the image. A specialized deep network allows us to find the most informative patches of a given size, and serves as an experimental tool. A human vision study recently characterized related (but different) minimally recognizable configurations (MIRCs) [1], for which we specify computational analogues (denoted cMIRCs). The drop in human decision accuracy associated with size reduction of these MIRCs is substantial and sharp. Interestingly, such sharp reductions were also found for the computational versions we specified.