25 years of CNNs: Can we compare to human abstraction capabilities?

TL;DR

This paper evaluates the progress of convolutional neural networks over 25 years in classifying abstract images, revealing that current models still lag behind human capabilities in solving certain vision tasks.

Contribution

It introduces a comparative framework using abstract image classification to assess the progress of CNNs versus human performance over time.

Findings

CNNs perform worse than humans on abstract classification tasks.

No significant progress in CNNs' ability to handle abstract visual properties over 25 years.

Current CNNs still struggle with tasks humans find easy.

Abstract

We try to determine the progress made by convolutional neural networks over the past 25 years in classifying images into abstractc lasses. For this purpose we compare the performance of LeNet to that of GoogLeNet at classifying randomly generated images which are differentiated by an abstract property (e.g., one class contains two objects of the same size, the other class two objects of different sizes). Our results show that there is still work to do in order to solve vision problems humans are able to solve without much difficulty.