From Concrete to Abstract: A Multimodal Generative Approach to Abstract Concept Learning

TL;DR

This paper presents a multimodal generative model that learns high-level abstract concepts by integrating visual and linguistic data, demonstrating strong language understanding and naming capabilities.

Contribution

It introduces a novel approach that grounds concrete concepts and progressively abstracts to higher-level concepts using multimodal data, advancing AI's understanding of abstract ideas.

Findings

Model effectively grounds concrete concepts

Successfully abstracts to superordinate concepts

Performs well in language understanding and naming tasks

Abstract

Understanding and manipulating concrete and abstract concepts is fundamental to human intelligence. Yet, they remain challenging for artificial agents. This paper introduces a multimodal generative approach to high order abstract concept learning, which integrates visual and categorical linguistic information from concrete ones. Our model initially grounds subordinate level concrete concepts, combines them to form basic level concepts, and finally abstracts to superordinate level concepts via the grounding of basic-level concepts. We evaluate the model language learning ability through language-to-visual and visual-to-language tests with high order abstract concepts. Experimental results demonstrate the proficiency of the model in both language understanding and language naming tasks.