What is Learning? A primary discussion about information and Representation

TL;DR

This paper explores the fundamental concept of learning in AI by analyzing the relationship between information and its representation, proposing a necessary condition for models to be considered capable of learning.

Contribution

It introduces a formal necessary condition for learning models based on information and representation, aiding in assessing a system’s ability to learn.

Findings

Proposes a necessary condition linking information and representation for learning

Provides a framework to verify if a system can learn

Enhances understanding of learning as a property of intelligence

Abstract

Nowadays, represented by Deep Learning techniques, the field of machine learning is experiencing unprecedented prosperity and its influence is demonstrated in academia, industry and civil society. "Intelligent" has become a label which could not be neglected for most applications; celebrities and scientists also warned that the development of full artificial intelligence may spell the end of the human race. It seems that the answer to building a computer system that could automatically improve with experience is right on the next corner. While for AI and machine learning researchers, it is a consensus that we are not anywhere near the core technique which could bring the Terminator, Number 5 or R2D2 into real life, and there is not even a formal definition about what is intelligence, or one of its basic properties: Learning. Therefore, even though researchers know these concerns are not necessary currently, there is no generalized explanation about why these concerns are not necessary, and what properties people should take into account that would make these concerns to be necessary. In this paper, starts from analysing the relation between information and its representation, a necessary condition for a model to be a learning model is proposed. This condition and related future works could be used to verify whether a system is able to learn or not, and enrich our understanding of learning: one important property of Intelligence.