What Is Considered Complete for Visual Recognition?

TL;DR

This paper argues that current visual recognition systems are incomplete and proposes a new pre-training approach called learning-by-compression, focusing on data representation and recovery rather than just accuracy.

Contribution

It introduces the concept of learning-by-compression as a new pre-training task for visual recognition, emphasizing data compression and recovery over traditional accuracy metrics.

Findings

Current systems are far from recognizing all that humans can.

Learning-by-compression can potentially bridge the recognition gap.

Evaluation of image recovery remains a key challenge.

Abstract

This is an opinion paper. We hope to deliver a key message that current visual recognition systems are far from complete, i.e., recognizing everything that human can recognize, yet it is very unlikely that the gap can be bridged by continuously increasing human annotations. Based on the observation, we advocate for a new type of pre-training task named learning-by-compression. The computational models (e.g., a deep network) are optimized to represent the visual data using compact features, and the features preserve the ability to recover the original data. Semantic annotations, when available, play the role of weak supervision. An important yet challenging issue is the evaluation of image recovery, where we suggest some design principles and future research directions. We hope our proposal can inspire the community to pursue the compression-recovery tradeoff rather than the accuracy-complexity tradeoff.