Concepts, Properties and an Approach for Compositional Generalization

TL;DR

This paper discusses the concept of compositional generalization, its importance in AI, and proposes an approach involving architecture design and regularization to improve neural networks' ability to recognize and generate novel combinations from known components.

Contribution

It introduces a framework connecting concepts and properties of compositional generalization and proposes an approach using architecture design and regularization to enhance this ability in neural networks.

Findings

Clarifies fundamental concepts and properties of compositional generalization

Proposes an architecture and regularization approach to improve generalization

Aims to advance understanding and development of AI systems with compositional skills

Abstract

Compositional generalization is the capacity to recognize and imagine a large amount of novel combinations from known components. It is a key in human intelligence, but current neural networks generally lack such ability. This report connects a series of our work for compositional generalization, and summarizes an approach. The first part contains concepts and properties. The second part looks into a machine learning approach. The approach uses architecture design and regularization to regulate information of representations. This report focuses on basic ideas with intuitive and illustrative explanations. We hope this work would be helpful to clarify fundamentals of compositional generalization and lead to advance artificial intelligence.