A Short Survey of Systematic Generalization

TL;DR

This survey reviews the concept, history, and recent advances in systematic generalization across AI fields, highlighting key approaches, challenges, and future directions.

Contribution

It provides a comprehensive overview of systematic generalization, categorizing methods and discussing core problems like variable binding and causality.

Findings

Systematic generalization is crucial across language, vision, and VQA.

Recent methods show progress but face fundamental challenges.

The survey offers a structured background for future research.

Abstract

This survey includes systematic generalization and a history of how machine learning addresses it. We aim to summarize and organize the related information of both conventional and recent improvements. We first look at the definition of systematic generalization, then introduce Classicist and Connectionist. We then discuss different types of Connectionists and how they approach the generalization. Two crucial problems of variable binding and causality are discussed. We look into systematic generalization in language, vision, and VQA fields. Recent improvements from different aspects are discussed. Systematic generalization has a long history in artificial intelligence. We could cover only a small portion of many contributions. We hope this paper provides a background and is beneficial for discoveries in future work.