Evolving Culture vs Local Minima
Yoshua Bengio

TL;DR
This paper presents a theory linking the difficulty of learning in deep neural architectures to human culture and language, suggesting that social interactions and language facilitate the learning of complex high-level abstractions.
Contribution
It introduces a novel theory connecting deep learning challenges with cultural evolution and language, providing a new perspective on human cognitive development.
Findings
Learning in humans is hindered by local minima in neural representations.
Language and social cues help humans overcome learning difficulties in high-level abstractions.
Cultural evolution accelerates the development of complex mental representations.
Abstract
We propose a theory that relates difficulty of learning in deep architectures to culture and language. It is articulated around the following hypotheses: (1) learning in an individual human brain is hampered by the presence of effective local minima; (2) this optimization difficulty is particularly important when it comes to learning higher-level abstractions, i.e., concepts that cover a vast and highly-nonlinear span of sensory configurations; (3) such high-level abstractions are best represented in brains by the composition of many levels of representation, i.e., by deep architectures; (4) a human brain can learn such high-level abstractions if guided by the signals produced by other humans, which act as hints or indirect supervision for these high-level abstractions; and (5), language and the recombination and optimization of mental concepts provide an efficient evolutionary…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsLanguage and cultural evolution · Evolutionary Algorithms and Applications · Cognitive Science and Education Research
