The Consciousness Prior
Yoshua Bengio

TL;DR
The paper introduces a consciousness-inspired prior for learning high-level concept representations, promoting disentanglement and aligning with cognitive neuroscience theories of attention and conscious states.
Contribution
It proposes a novel prior based on consciousness theories, enabling better disentanglement of abstract factors and integration with language-like representations.
Findings
Supports sparse factor graph structure for high-level concepts
Aligns conscious states with natural language expressions
Facilitates disentangling abstract factors in representation learning
Abstract
A new prior is proposed for learning representations of high-level concepts of the kind we manipulate with language. This prior can be combined with other priors in order to help disentangling abstract factors from each other. It is inspired by cognitive neuroscience theories of consciousness, seen as a bottleneck through which just a few elements, after having been selected by attention from a broader pool, are then broadcast and condition further processing, both in perception and decision-making. The set of recently selected elements one becomes aware of is seen as forming a low-dimensional conscious state. This conscious state is combining the few concepts constituting a conscious thought, i.e., what one is immediately conscious of at a particular moment. We claim that this architectural and information-processing constraint corresponds to assumptions about the joint distribution…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Neural Networks and Applications · Machine Learning and Algorithms
